[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:49:56 dist-test-slave-vqch krb5kdc[20810](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:49:56 dist-test-slave-vqch krb5kdc[20810](info): set up 2 sockets
Sep 29 10:49:56 dist-test-slave-vqch krb5kdc[20810](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260929 10:49:57.187510 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/krb5kdc/test-user.keytab.
W20260929 10:49:57.759063 20803 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.419s user 0.000s sys 0.001s
Sep 29 10:49:57 dist-test-slave-vqch krb5kdc[20810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:49:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:49:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.80.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
I20260929 10:49:57.804946 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:46495
--webserver_interface=127.20.80.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:44711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:46495
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:49:57.887192 20823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:49:57.887401 20823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:49:57.887423 20823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:49:57.888589 20823 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:49:57.888633 20823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:49:57.888651 20823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:49:57.888702 20823 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:49:57.888722 20823 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:49:57.890235 20823 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:44711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:46495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:46495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:49:57.890486 20823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:49:57.890725 20823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 10:49:57.895982 20823 server_base.cc:1034] running on GCE node
W20260929 10:49:57.895967 20828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:49:57.896000 20831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:49:57.896011 20829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:49:57.896489 20823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:49:57.896847 20823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:49:57.898023 20823 hybrid_clock.cc:648] HybridClock initialized: now 1790678997898008 us; error 45 us; skew 500 ppm
Sep 29 10:49:57 dist-test-slave-vqch krb5kdc[20810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678997, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:49:57.900624 20823 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:49:57.901188 20823 webserver.cc:460] Webserver started at http://127.20.80.254:38143/ using document root <none> and password file <none>
I20260929 10:49:57.901425 20823 fs_manager.cc:359] Metadata directory not provided
I20260929 10:49:57.901494 20823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:49:57.901659 20823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:49:57.902758 20823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "fd07d24f87594309aea1507904c9813a"
format_stamp: "Formatted at 2026-09-29 10:49:57 on dist-test-slave-vqch"
server_key: "89834a9c10bf7b2acb40eae792d28801"
server_key_iv: "2c95f820fc9c2958dcd969e539c27100"
server_key_version: "encryptionkey@0"
I20260929 10:49:57.903172 20823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "fd07d24f87594309aea1507904c9813a"
format_stamp: "Formatted at 2026-09-29 10:49:57 on dist-test-slave-vqch"
server_key: "89834a9c10bf7b2acb40eae792d28801"
server_key_iv: "2c95f820fc9c2958dcd969e539c27100"
server_key_version: "encryptionkey@0"
I20260929 10:49:57.904476 20823 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:49:57.905298 20838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:49:57.905505 20823 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:49:57.905660 20823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "fd07d24f87594309aea1507904c9813a"
format_stamp: "Formatted at 2026-09-29 10:49:57 on dist-test-slave-vqch"
server_key: "89834a9c10bf7b2acb40eae792d28801"
server_key_iv: "2c95f820fc9c2958dcd969e539c27100"
server_key_version: "encryptionkey@0"
I20260929 10:49:57.905758 20823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:49:57.915102 20823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:49:57.917732 20823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:49:57.917904 20823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:49:57.922207 20823 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:46495
I20260929 10:49:57.922237 20890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:46495 every 8 connection(s)
I20260929 10:49:57.922601 20823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:49:57.923475 20891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:49:57.925815 20891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd07d24f87594309aea1507904c9813a: Bootstrap starting.
I20260929 10:49:57.926362 20891 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd07d24f87594309aea1507904c9813a: Neither blocks nor log segments found. Creating new log.
I20260929 10:49:57.926610 20891 log.cc:824] T 00000000000000000000000000000000 P fd07d24f87594309aea1507904c9813a: Log is configured to *not* fsync() on all Append() calls
I20260929 10:49:57.927306 20891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd07d24f87594309aea1507904c9813a: No bootstrap required, opened a new log
I20260929 10:49:57.929114 20891 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd07d24f87594309aea1507904c9813a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd07d24f87594309aea1507904c9813a" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46495 } }
I20260929 10:49:57.929252 20891 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd07d24f87594309aea1507904c9813a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:49:57.929280 20891 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd07d24f87594309aea1507904c9813a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd07d24f87594309aea1507904c9813a, State: Initialized, Role: FOLLOWER
I20260929 10:49:57.929409 20891 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd07d24f87594309aea1507904c9813a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd07d24f87594309aea1507904c9813a" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46495 } }
I20260929 10:49:57.929493 20891 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd07d24f87594309aea1507904c9813a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:49:57.929517 20891 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd07d24f87594309aea1507904c9813a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:49:57.929580 20891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd07d24f87594309aea1507904c9813a [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:49:57.929910 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 20823
I20260929 10:49:57.930015 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:49:57.930303 20891 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd07d24f87594309aea1507904c9813a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd07d24f87594309aea1507904c9813a" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46495 } }
I20260929 10:49:57.930413 20891 leader_election.cc:302] T 00000000000000000000000000000000 P fd07d24f87594309aea1507904c9813a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd07d24f87594309aea1507904c9813a; no voters:
I20260929 10:49:57.930567 20891 leader_election.cc:288] T 00000000000000000000000000000000 P fd07d24f87594309aea1507904c9813a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:49:57.930620 20894 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd07d24f87594309aea1507904c9813a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:49:57.930807 20894 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd07d24f87594309aea1507904c9813a [term 1 LEADER]: Becoming Leader. State: Replica: fd07d24f87594309aea1507904c9813a, State: Running, Role: LEADER
I20260929 10:49:57.930832 20803 external_mini_cluster.cc:1428] Setting key a3a960b63a955100e16ac0cdb8f8a22b
I20260929 10:49:57.930817 20891 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd07d24f87594309aea1507904c9813a [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:49:57.930926 20894 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd07d24f87594309aea1507904c9813a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd07d24f87594309aea1507904c9813a" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46495 } }
I20260929 10:49:57.931741 20895 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd07d24f87594309aea1507904c9813a [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd07d24f87594309aea1507904c9813a. Latest consensus state: current_term: 1 leader_uuid: "fd07d24f87594309aea1507904c9813a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd07d24f87594309aea1507904c9813a" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46495 } } }
I20260929 10:49:57.931847 20895 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd07d24f87594309aea1507904c9813a [sys.catalog]: This master's current role is: LEADER
I20260929 10:49:57.932220 20896 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd07d24f87594309aea1507904c9813a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd07d24f87594309aea1507904c9813a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd07d24f87594309aea1507904c9813a" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46495 } } }
I20260929 10:49:57.932339 20896 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd07d24f87594309aea1507904c9813a [sys.catalog]: This master's current role is: LEADER
W20260929 10:49:57.932907 20912 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fd07d24f87594309aea1507904c9813a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:49:57.932972 20912 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:49:57.934743 20904 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:49:57.934922 20904 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:49:57.936940 20904 catalog_manager.cc:1347] Generated new cluster ID: 7626afe185924841bef244d3123cc0d8
I20260929 10:49:57.937012 20904 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 10:49:57 dist-test-slave-vqch krb5kdc[20810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:49:57.951572 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:49:57.932153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48462 (local address 127.20.80.254:46495)
0929 10:49:57.936034 (+ 3881us) server_negotiation.cc:206] Beginning negotiation
0929 10:49:57.936043 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:49:57.936075 (+ 32us) server_negotiation.cc:407] Connection header received
0929 10:49:57.936223 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:49:57.936229 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:49:57.936303 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:49:57.936446 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:49:57.939724 (+ 3278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:49:57.941290 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:49:57.942903 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:49:57.943544 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:49:57.943628 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:49:57.947565 (+ 3937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:49:57.947570 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:49:57.947570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:49:57.947580 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:49:57.949634 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:49:57.950079 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:49:57.950082 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:49:57.950082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:49:57.950176 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:49:57.950455 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:49:57.950456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:49:57.950456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:49:57.950713 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:49:57.950859 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0929 10:49:57.951169 (+ 310us) server_negotiation.cc:299] Negotiation successful
0929 10:49:57.951309 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3837,"thread_start_us":83,"threads_started":1}
I20260929 10:49:57.982615 20904 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:49:57.983373 20904 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:49:57.993166 20904 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd07d24f87594309aea1507904c9813a: Generated new TSK 0
I20260929 10:49:57.993381 20904 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.80.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
I20260929 10:49:58.020586 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.193:0
--local_ip_for_outbound_sockets=127.20.80.193
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:46495
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:44711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:49:58.101214 20917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:49:58.101445 20917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:49:58.101481 20917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:49:58.102602 20917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:49:58.102669 20917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:49:58.102711 20917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.193
I20260929 10:49:58.104163 20917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:44711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:46495
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.193
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:49:58.104440 20917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:49:58.104722 20917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:49:58.109647 20923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:49:58.109670 20922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:49:58.109833 20925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:49:58.109817 20917 server_base.cc:1034] running on GCE node
I20260929 10:49:58.110083 20917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:49:58.110328 20917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:49:58.111490 20917 hybrid_clock.cc:648] HybridClock initialized: now 1790678998111480 us; error 42 us; skew 500 ppm
Sep 29 10:49:58 dist-test-slave-vqch krb5kdc[20810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678998, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:49:58.114107 20917 init.cc:377] Logged in from keytab as kudu/127.20.80.193@KRBTEST.COM (short username kudu)
I20260929 10:49:58.114530 20917 webserver.cc:460] Webserver started at http://127.20.80.193:42135/ using document root <none> and password file <none>
I20260929 10:49:58.114764 20917 fs_manager.cc:359] Metadata directory not provided
I20260929 10:49:58.114841 20917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:49:58.114985 20917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:49:58.116040 20917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-0/data/instance:
uuid: "7455fec7d1b44ed98790461f4ed7c1a9"
format_stamp: "Formatted at 2026-09-29 10:49:58 on dist-test-slave-vqch"
server_key: "8bffef0e023aca5efc1f4f67e9b6dd58"
server_key_iv: "8132b5e2e97f7146de5272929f067097"
server_key_version: "encryptionkey@0"
I20260929 10:49:58.116400 20917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance:
uuid: "7455fec7d1b44ed98790461f4ed7c1a9"
format_stamp: "Formatted at 2026-09-29 10:49:58 on dist-test-slave-vqch"
server_key: "8bffef0e023aca5efc1f4f67e9b6dd58"
server_key_iv: "8132b5e2e97f7146de5272929f067097"
server_key_version: "encryptionkey@0"
I20260929 10:49:58.117844 20917 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 10:49:58.118619 20932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:49:58.118841 20917 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:49:58.118944 20917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-0/wal
uuid: "7455fec7d1b44ed98790461f4ed7c1a9"
format_stamp: "Formatted at 2026-09-29 10:49:58 on dist-test-slave-vqch"
server_key: "8bffef0e023aca5efc1f4f67e9b6dd58"
server_key_iv: "8132b5e2e97f7146de5272929f067097"
server_key_version: "encryptionkey@0"
I20260929 10:49:58.119023 20917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:49:58.133906 20917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:49:58.136284 20917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:49:58.136451 20917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:49:58.136695 20917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:49:58.137128 20917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:49:58.137185 20917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:49:58.137225 20917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:49:58.137253 20917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:49:58.143062 20917 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.193:42857
I20260929 10:49:58.143112 21045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.193:42857 every 8 connection(s)
I20260929 10:49:58.143487 20917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
I20260929 10:49:58.145943 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 20917
I20260929 10:49:58.146303 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance
I20260929 10:49:58.146900 20803 external_mini_cluster.cc:1428] Setting key a1d5c5242810e074d635654dc39cf772
Sep 29 10:49:58 dist-test-slave-vqch krb5kdc[20810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678998, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.80.194@KRBTEST.COM; defaulting to no policy
I20260929 10:49:58.158737 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:49:58.144294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:46197 (local address 127.20.80.254:46495)
0929 10:49:58.144381 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0929 10:49:58.144383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:49:58.144564 (+ 181us) server_negotiation.cc:407] Connection header received
0929 10:49:58.145178 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:49:58.145180 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:49:58.145195 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:49:58.145275 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:49:58.146880 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:49:58.147613 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:49:58.148917 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:49:58.149377 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:49:58.149560 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:49:58.152968 (+ 3408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:49:58.152970 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:49:58.152971 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:49:58.153136 (+ 165us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:49:58.155943 (+ 2807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:49:58.156594 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:49:58.156596 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:49:58.156596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:49:58.156672 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:49:58.157201 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:49:58.157204 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:49:58.157204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:49:58.157397 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:49:58.157492 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0929 10:49:58.158497 (+ 1005us) server_negotiation.cc:299] Negotiation successful
0929 10:49:58.158636 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 10:49:58.158972 21048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:49:58.144248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:46495 (local address 127.20.80.193:46197)
0929 10:49:58.144429 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 10:49:58.144474 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 10:49:58.144966 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:49:58.145375 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:49:58.145379 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:49:58.146163 (+ 784us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:49:58.146448 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:49:58.146455 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:49:58.147826 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:49:58.147827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:49:58.148699 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:49:58.148702 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:49:58.149617 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:49:58.149619 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:49:58.149830 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:49:58.150661 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:49:58.150675 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:49:58.152799 (+ 2124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:49:58.156096 (+ 3297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:49:58.156122 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:49:58.156126 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:49:58.156477 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:49:58.156816 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:49:58.156817 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:49:58.156818 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:49:58.157078 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:49:58.157756 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:49:58.157787 (+ 31us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:49:58.158231 (+ 444us) client_negotiation.cc:769] Sending connection context
0929 10:49:58.158396 (+ 165us) client_negotiation.cc:240] Negotiation successful
0929 10:49:58.158599 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":89,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.194@KRBTEST.COM" created.
I20260929 10:49:58.159967 21046 heartbeater.cc:347] Connected to a master server at 127.20.80.254:46495
I20260929 10:49:58.160141 21046 heartbeater.cc:464] Registering TS with master...
I20260929 10:49:58.160360 21046 heartbeater.cc:511] Master 127.20.80.254:46495 requested a full tablet report, sending...
I20260929 10:49:58.161201 20855 ts_manager.cc:194] Registered new tserver with Master: 7455fec7d1b44ed98790461f4ed7c1a9 (127.20.80.193:42857)
I20260929 10:49:58.161909 20855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.193@KRBTEST.COM'} at 127.20.80.193:46197
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
I20260929 10:49:58.181249 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.194:0
--local_ip_for_outbound_sockets=127.20.80.194
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:46495
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:44711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:49:58.262174 21051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:49:58.262413 21051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:49:58.262450 21051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:49:58.263566 21051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:49:58.263633 21051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:49:58.263676 21051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.194
I20260929 10:49:58.265163 21051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:44711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:46495
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.194
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:49:58.265419 21051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:49:58.265636 21051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:49:58.270594 21057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:49:58.270625 21051 server_base.cc:1034] running on GCE node
W20260929 10:49:58.270625 21056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:49:58.270762 21059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:49:58.270991 21051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:49:58.271183 21051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:49:58.272327 21051 hybrid_clock.cc:648] HybridClock initialized: now 1790678998272302 us; error 46 us; skew 500 ppm
Sep 29 10:49:58 dist-test-slave-vqch krb5kdc[20810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678998, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:49:58.274773 21051 init.cc:377] Logged in from keytab as kudu/127.20.80.194@KRBTEST.COM (short username kudu)
I20260929 10:49:58.275143 21051 webserver.cc:460] Webserver started at http://127.20.80.194:36059/ using document root <none> and password file <none>
I20260929 10:49:58.275326 21051 fs_manager.cc:359] Metadata directory not provided
I20260929 10:49:58.275368 21051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:49:58.275452 21051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:49:58.276476 21051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-1/data/instance:
uuid: "32241c4615a44ec7a36d43b32e0962f9"
format_stamp: "Formatted at 2026-09-29 10:49:58 on dist-test-slave-vqch"
server_key: "a1959d54f5d43e06c4e4f76d4e52a787"
server_key_iv: "79fee7fa05313e9f74553a838dc867b7"
server_key_version: "encryptionkey@0"
I20260929 10:49:58.276788 21051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance:
uuid: "32241c4615a44ec7a36d43b32e0962f9"
format_stamp: "Formatted at 2026-09-29 10:49:58 on dist-test-slave-vqch"
server_key: "a1959d54f5d43e06c4e4f76d4e52a787"
server_key_iv: "79fee7fa05313e9f74553a838dc867b7"
server_key_version: "encryptionkey@0"
I20260929 10:49:58.278103 21051 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 10:49:58.278797 21066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:49:58.278977 21051 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:49:58.279085 21051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-1/wal
uuid: "32241c4615a44ec7a36d43b32e0962f9"
format_stamp: "Formatted at 2026-09-29 10:49:58 on dist-test-slave-vqch"
server_key: "a1959d54f5d43e06c4e4f76d4e52a787"
server_key_iv: "79fee7fa05313e9f74553a838dc867b7"
server_key_version: "encryptionkey@0"
I20260929 10:49:58.279177 21051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:49:58.301083 21051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:49:58.303483 21051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:49:58.303650 21051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:49:58.303917 21051 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:49:58.304240 21051 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:49:58.304291 21051 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:49:58.304316 21051 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:49:58.304358 21051 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:49:58.310477 21051 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.194:38769
I20260929 10:49:58.310515 21179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.194:38769 every 8 connection(s)
I20260929 10:49:58.310845 21051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
I20260929 10:49:58.316377 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 21051
I20260929 10:49:58.316457 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance
I20260929 10:49:58.316679 20803 external_mini_cluster.cc:1428] Setting key 8bbfb77edffe142ceecedd4764788dad
Sep 29 10:49:58 dist-test-slave-vqch krb5kdc[20810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678998, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:49:58.321116 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:49:58.311610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:47243 (local address 127.20.80.254:46495)
0929 10:49:58.311693 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0929 10:49:58.311695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:49:58.311919 (+ 224us) server_negotiation.cc:407] Connection header received
0929 10:49:58.312472 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:49:58.312473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:49:58.312487 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:49:58.312546 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:49:58.313539 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:49:58.314084 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:49:58.315189 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:49:58.315411 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:49:58.315490 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:49:58.317953 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:49:58.317954 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:49:58.317954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:49:58.317962 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:49:58.319632 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:49:58.320036 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:49:58.320037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:49:58.320037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:49:58.320099 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:49:58.320343 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:49:58.320344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:49:58.320345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:49:58.320504 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:49:58.320573 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 10:49:58.320813 (+ 240us) server_negotiation.cc:299] Negotiation successful
0929 10:49:58.320918 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 10:49:58.321128 21182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:49:58.311585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:46495 (local address 127.20.80.194:47243)
0929 10:49:58.311794 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0929 10:49:58.311831 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 10:49:58.312351 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:49:58.312621 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:49:58.312624 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:49:58.313226 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:49:58.313425 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:49:58.313430 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:49:58.314190 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:49:58.314191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:49:58.315082 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:49:58.315085 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:49:58.315495 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:49:58.315496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:49:58.315631 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:49:58.316140 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:49:58.316150 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:49:58.317840 (+ 1690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:49:58.319734 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:49:58.319739 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:49:58.319741 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:49:58.319965 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:49:58.320170 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:49:58.320171 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:49:58.320171 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:49:58.320263 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:49:58.320569 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:49:58.320573 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:49:58.320688 (+ 115us) client_negotiation.cc:769] Sending connection context
0929 10:49:58.320769 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 10:49:58.320888 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":115,"threads_started":1}
I20260929 10:49:58.321666 21180 heartbeater.cc:347] Connected to a master server at 127.20.80.254:46495
I20260929 10:49:58.321779 21180 heartbeater.cc:464] Registering TS with master...
I20260929 10:49:58.321990 21180 heartbeater.cc:511] Master 127.20.80.254:46495 requested a full tablet report, sending...
I20260929 10:49:58.322376 20855 ts_manager.cc:194] Registered new tserver with Master: 32241c4615a44ec7a36d43b32e0962f9 (127.20.80.194:38769)
I20260929 10:49:58.322811 20855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.194@KRBTEST.COM'} at 127.20.80.194:47243
WARNING: no policy specified for kudu/127.20.80.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
I20260929 10:49:58.343540 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.195:0
--local_ip_for_outbound_sockets=127.20.80.195
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:46495
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:44711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:49:58.424655 21185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:49:58.424902 21185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:49:58.424926 21185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:49:58.426296 21185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:49:58.426357 21185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:49:58.426388 21185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.195
I20260929 10:49:58.427831 21185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:44711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:46495
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.195
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:49:58.428030 21185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:49:58.428212 21185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:49:58.433283 21193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:49:58.433365 21190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:49:58.433442 21185 server_base.cc:1034] running on GCE node
W20260929 10:49:58.433286 21191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:49:58.433759 21185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:49:58.434031 21185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:49:58.435190 21185 hybrid_clock.cc:648] HybridClock initialized: now 1790678998435189 us; error 38 us; skew 500 ppm
Sep 29 10:49:58 dist-test-slave-vqch krb5kdc[20810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678998, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:49:58.437731 21185 init.cc:377] Logged in from keytab as kudu/127.20.80.195@KRBTEST.COM (short username kudu)
I20260929 10:49:58.438166 21185 webserver.cc:460] Webserver started at http://127.20.80.195:44955/ using document root <none> and password file <none>
I20260929 10:49:58.438395 21185 fs_manager.cc:359] Metadata directory not provided
I20260929 10:49:58.438463 21185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:49:58.438613 21185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:49:58.439635 21185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-2/data/instance:
uuid: "6dae42206f864da594219fa4cc09d6a3"
format_stamp: "Formatted at 2026-09-29 10:49:58 on dist-test-slave-vqch"
server_key: "1830905fc2e56f376338ac24d91c9e4b"
server_key_iv: "226d845e856af489142f8ff491b4b9d4"
server_key_version: "encryptionkey@0"
I20260929 10:49:58.440009 21185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance:
uuid: "6dae42206f864da594219fa4cc09d6a3"
format_stamp: "Formatted at 2026-09-29 10:49:58 on dist-test-slave-vqch"
server_key: "1830905fc2e56f376338ac24d91c9e4b"
server_key_iv: "226d845e856af489142f8ff491b4b9d4"
server_key_version: "encryptionkey@0"
I20260929 10:49:58.441392 21185 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260929 10:49:58.442131 21200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:49:58.442317 21185 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:49:58.442415 21185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-2/wal
uuid: "6dae42206f864da594219fa4cc09d6a3"
format_stamp: "Formatted at 2026-09-29 10:49:58 on dist-test-slave-vqch"
server_key: "1830905fc2e56f376338ac24d91c9e4b"
server_key_iv: "226d845e856af489142f8ff491b4b9d4"
server_key_version: "encryptionkey@0"
I20260929 10:49:58.442492 21185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:49:58.474644 21185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:49:58.477152 21185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:49:58.477325 21185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:49:58.477578 21185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:49:58.477901 21185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:49:58.477957 21185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:49:58.477995 21185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:49:58.478024 21185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:49:58.483930 21185 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.195:44157
I20260929 10:49:58.483983 21313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.195:44157 every 8 connection(s)
I20260929 10:49:58.484282 21185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
I20260929 10:49:58.488320 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 21185
I20260929 10:49:58.488425 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance
I20260929 10:49:58.488576 20803 external_mini_cluster.cc:1428] Setting key 321aba75e8cf451d4912860ef336b461
Sep 29 10:49:58 dist-test-slave-vqch krb5kdc[20810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678998, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:49:58.493717 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:49:58.485058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:42243 (local address 127.20.80.254:46495)
0929 10:49:58.485119 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 10:49:58.485121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:49:58.485316 (+ 195us) server_negotiation.cc:407] Connection header received
0929 10:49:58.485852 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:49:58.485853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:49:58.485865 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:49:58.485922 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:49:58.486858 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:49:58.487366 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:49:58.488371 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:49:58.488641 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:49:58.488713 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:49:58.490997 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:49:58.490998 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:49:58.490999 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:49:58.491005 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:49:58.492204 (+ 1199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:49:58.492629 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:49:58.492629 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:49:58.492630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:49:58.492685 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:49:58.493015 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:49:58.493016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:49:58.493017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:49:58.493174 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:49:58.493241 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 10:49:58.493526 (+ 285us) server_negotiation.cc:299] Negotiation successful
0929 10:49:58.493611 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 10:49:58.493923 21316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:49:58.485036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:46495 (local address 127.20.80.195:42243)
0929 10:49:58.485195 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0929 10:49:58.485230 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 10:49:58.485741 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:49:58.485991 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:49:58.485996 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:49:58.486568 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:49:58.486758 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:49:58.486763 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:49:58.487467 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:49:58.487468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:49:58.488259 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:49:58.488261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:49:58.488766 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:49:58.488767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:49:58.488916 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:49:58.489506 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:49:58.489515 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:49:58.490895 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:49:58.492314 (+ 1419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:49:58.492320 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:49:58.492322 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:49:58.492557 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:49:58.492792 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:49:58.492793 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:49:58.492794 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:49:58.492901 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:49:58.493243 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:49:58.493250 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:49:58.493384 (+ 134us) client_negotiation.cc:769] Sending connection context
0929 10:49:58.493490 (+ 106us) client_negotiation.cc:240] Negotiation successful
0929 10:49:58.493680 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":73,"threads_started":1}
I20260929 10:49:58.494396 21314 heartbeater.cc:347] Connected to a master server at 127.20.80.254:46495
I20260929 10:49:58.494477 21314 heartbeater.cc:464] Registering TS with master...
I20260929 10:49:58.494678 21314 heartbeater.cc:511] Master 127.20.80.254:46495 requested a full tablet report, sending...
I20260929 10:49:58.495054 20855 ts_manager.cc:194] Registered new tserver with Master: 6dae42206f864da594219fa4cc09d6a3 (127.20.80.195:44157)
I20260929 10:49:58.495450 20855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.195@KRBTEST.COM'} at 127.20.80.195:42243
I20260929 10:49:58.500444 20803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 10:49:58 dist-test-slave-vqch krb5kdc[20810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678998, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 10:49:58 dist-test-slave-vqch krb5kdc[20810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678998, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.80.193@KRBTEST.COM
I20260929 10:49:58.523360 21321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:49:58.515326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33576 (local address 127.20.80.193:42857)
0929 10:49:58.515542 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0929 10:49:58.515546 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:49:58.515559 (+ 13us) server_negotiation.cc:407] Connection header received
0929 10:49:58.515588 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:49:58.515589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:49:58.515606 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:49:58.515687 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:49:58.516250 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:49:58.517072 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:49:58.517844 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:49:58.518215 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:49:58.518282 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:49:58.520573 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:49:58.520581 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:49:58.520581 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:49:58.520588 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:49:58.522335 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:49:58.522661 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:49:58.522664 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:49:58.522665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:49:58.522733 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:49:58.522886 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:49:58.522887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:49:58.522887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:49:58.523003 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:49:58.523130 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0929 10:49:58.523164 (+ 34us) server_negotiation.cc:299] Negotiation successful
0929 10:49:58.523223 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":108,"threads_started":1}
W20260929 10:49:58.523692 20960 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33576
Sep 29 10:49:58 dist-test-slave-vqch krb5kdc[20810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 10:49:58 dist-test-slave-vqch krb5kdc[20810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.193@KRBTEST.COM
I20260929 10:49:58.545868 21321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:49:58.538609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33586 (local address 127.20.80.193:42857)
0929 10:49:58.538701 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0929 10:49:58.538703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:49:58.538784 (+ 81us) server_negotiation.cc:407] Connection header received
0929 10:49:58.538830 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:49:58.538830 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:49:58.538841 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:49:58.538890 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:49:58.539455 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:49:58.539927 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:49:58.540662 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:49:58.540858 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:49:58.540911 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:49:58.543127 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:49:58.543128 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:49:58.543129 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:49:58.543135 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:49:58.544573 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:49:58.545240 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:49:58.545241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:49:58.545241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:49:58.545294 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:49:58.545460 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:49:58.545460 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:49:58.545460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:49:58.545542 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:49:58.545629 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0929 10:49:58.545704 (+ 75us) server_negotiation.cc:299] Negotiation successful
0929 10:49:58.545765 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 10:49:58.546600 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 20917
I20260929 10:49:58.552387 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 21051
I20260929 10:49:58.558403 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 21185
I20260929 10:49:58.562356 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 20823
2026-09-29T10:49:58Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2443 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:49:58 dist-test-slave-vqch krb5kdc[21328](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:49:58 dist-test-slave-vqch krb5kdc[21328](info): set up 2 sockets
Sep 29 10:49:58 dist-test-slave-vqch krb5kdc[21328](info): commencing operation
krb5kdc: starting...
W20260929 10:49:59.604295 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:49:59 dist-test-slave-vqch krb5kdc[21328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:49:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:49:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.80.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
I20260929 10:49:59.724658 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:37351
--webserver_interface=127.20.80.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:33669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:37351
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:49:59.804517 21341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:49:59.804768 21341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:49:59.804814 21341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:49:59.806000 21341 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:49:59.806070 21341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:49:59.806095 21341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:49:59.806121 21341 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:49:59.806152 21341 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:49:59.807597 21341 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:33669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:37351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:37351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:49:59.807879 21341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:49:59.808104 21341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 10:49:59.813197 21341 server_base.cc:1034] running on GCE node
W20260929 10:49:59.813143 21347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:49:59.813150 21346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:49:59.813166 21349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:49:59.813594 21341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:49:59.813859 21341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:49:59.815001 21341 hybrid_clock.cc:648] HybridClock initialized: now 1790678999814975 us; error 49 us; skew 500 ppm
Sep 29 10:49:59 dist-test-slave-vqch krb5kdc[21328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678999, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:49:59.817632 21341 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:49:59.818135 21341 webserver.cc:460] Webserver started at http://127.20.80.254:43595/ using document root <none> and password file <none>
I20260929 10:49:59.818365 21341 fs_manager.cc:359] Metadata directory not provided
I20260929 10:49:59.818425 21341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:49:59.818588 21341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:49:59.819679 21341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "f421038f9d7049d6806440dabf875ab8"
format_stamp: "Formatted at 2026-09-29 10:49:59 on dist-test-slave-vqch"
server_key: "a8beacc14844f3415d95e12aab575c7e"
server_key_iv: "601c4729dcf50c17df1584cb16368808"
server_key_version: "encryptionkey@0"
I20260929 10:49:59.820066 21341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "f421038f9d7049d6806440dabf875ab8"
format_stamp: "Formatted at 2026-09-29 10:49:59 on dist-test-slave-vqch"
server_key: "a8beacc14844f3415d95e12aab575c7e"
server_key_iv: "601c4729dcf50c17df1584cb16368808"
server_key_version: "encryptionkey@0"
I20260929 10:49:59.821576 21341 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:49:59.822505 21356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:49:59.822820 21341 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:49:59.822966 21341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "f421038f9d7049d6806440dabf875ab8"
format_stamp: "Formatted at 2026-09-29 10:49:59 on dist-test-slave-vqch"
server_key: "a8beacc14844f3415d95e12aab575c7e"
server_key_iv: "601c4729dcf50c17df1584cb16368808"
server_key_version: "encryptionkey@0"
I20260929 10:49:59.823061 21341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:49:59.848173 21341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:49:59.850629 21341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:49:59.850821 21341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:49:59.855183 21341 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:37351
I20260929 10:49:59.855305 21408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:37351 every 8 connection(s)
I20260929 10:49:59.855588 21341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:49:59.856483 21409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:49:59.858888 21409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f421038f9d7049d6806440dabf875ab8: Bootstrap starting.
I20260929 10:49:59.859395 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 21341
I20260929 10:49:59.859464 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:49:59.859565 21409 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f421038f9d7049d6806440dabf875ab8: Neither blocks nor log segments found. Creating new log.
I20260929 10:49:59.859619 20803 external_mini_cluster.cc:1428] Setting key 829486eb626ed96b77bfcb00817d7654
I20260929 10:49:59.859905 21409 log.cc:824] T 00000000000000000000000000000000 P f421038f9d7049d6806440dabf875ab8: Log is configured to *not* fsync() on all Append() calls
I20260929 10:49:59.860635 21409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f421038f9d7049d6806440dabf875ab8: No bootstrap required, opened a new log
I20260929 10:49:59.862512 21409 raft_consensus.cc:348] T 00000000000000000000000000000000 P f421038f9d7049d6806440dabf875ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f421038f9d7049d6806440dabf875ab8" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37351 } }
I20260929 10:49:59.862670 21409 raft_consensus.cc:374] T 00000000000000000000000000000000 P f421038f9d7049d6806440dabf875ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:49:59.862733 21409 raft_consensus.cc:729] T 00000000000000000000000000000000 P f421038f9d7049d6806440dabf875ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f421038f9d7049d6806440dabf875ab8, State: Initialized, Role: FOLLOWER
I20260929 10:49:59.862874 21409 consensus_queue.cc:260] T 00000000000000000000000000000000 P f421038f9d7049d6806440dabf875ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f421038f9d7049d6806440dabf875ab8" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37351 } }
I20260929 10:49:59.862938 21409 raft_consensus.cc:388] T 00000000000000000000000000000000 P f421038f9d7049d6806440dabf875ab8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:49:59.862959 21409 raft_consensus.cc:482] T 00000000000000000000000000000000 P f421038f9d7049d6806440dabf875ab8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:49:59.863022 21409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f421038f9d7049d6806440dabf875ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:49:59.863529 21409 raft_consensus.cc:504] T 00000000000000000000000000000000 P f421038f9d7049d6806440dabf875ab8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f421038f9d7049d6806440dabf875ab8" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37351 } }
I20260929 10:49:59.863646 21409 leader_election.cc:302] T 00000000000000000000000000000000 P f421038f9d7049d6806440dabf875ab8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f421038f9d7049d6806440dabf875ab8; no voters:
I20260929 10:49:59.863842 21409 leader_election.cc:288] T 00000000000000000000000000000000 P f421038f9d7049d6806440dabf875ab8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:49:59.863889 21414 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f421038f9d7049d6806440dabf875ab8 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:49:59.864066 21414 raft_consensus.cc:686] T 00000000000000000000000000000000 P f421038f9d7049d6806440dabf875ab8 [term 1 LEADER]: Becoming Leader. State: Replica: f421038f9d7049d6806440dabf875ab8, State: Running, Role: LEADER
I20260929 10:49:59.864091 21409 sys_catalog.cc:564] T 00000000000000000000000000000000 P f421038f9d7049d6806440dabf875ab8 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:49:59.864218 21414 consensus_queue.cc:237] T 00000000000000000000000000000000 P f421038f9d7049d6806440dabf875ab8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f421038f9d7049d6806440dabf875ab8" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37351 } }
I20260929 10:49:59.864656 21414 sys_catalog.cc:455] T 00000000000000000000000000000000 P f421038f9d7049d6806440dabf875ab8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f421038f9d7049d6806440dabf875ab8. Latest consensus state: current_term: 1 leader_uuid: "f421038f9d7049d6806440dabf875ab8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f421038f9d7049d6806440dabf875ab8" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37351 } } }
I20260929 10:49:59.864758 21414 sys_catalog.cc:458] T 00000000000000000000000000000000 P f421038f9d7049d6806440dabf875ab8 [sys.catalog]: This master's current role is: LEADER
I20260929 10:49:59.864668 21415 sys_catalog.cc:455] T 00000000000000000000000000000000 P f421038f9d7049d6806440dabf875ab8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f421038f9d7049d6806440dabf875ab8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f421038f9d7049d6806440dabf875ab8" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37351 } } }
I20260929 10:49:59.864969 21415 sys_catalog.cc:458] T 00000000000000000000000000000000 P f421038f9d7049d6806440dabf875ab8 [sys.catalog]: This master's current role is: LEADER
I20260929 10:49:59.865001 21423 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:49:59.865561 21423 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 29 10:49:59 dist-test-slave-vqch krb5kdc[21328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:49:59.867265 21423 catalog_manager.cc:1347] Generated new cluster ID: 9303f2045e934af9928c43f9f71b06c6
I20260929 10:49:59.867317 21423 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:49:59.870901 21413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:49:59.860182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60616 (local address 127.20.80.254:37351)
0929 10:49:59.860363 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 10:49:59.860373 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 10:49:59.860411 (+ 38us) server_negotiation.cc:407] Connection header received
0929 10:49:59.860889 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:49:59.860896 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:49:59.860943 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:49:59.861129 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:49:59.861742 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:49:59.862631 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:49:59.863585 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:49:59.864298 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:49:59.865002 (+ 704us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:49:59.867517 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:49:59.867521 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:49:59.867521 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:49:59.867531 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:49:59.869246 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:49:59.869662 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:49:59.869665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:49:59.869665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:49:59.869766 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:49:59.870001 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:49:59.870002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:49:59.870002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:49:59.870298 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:49:59.870431 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0929 10:49:59.870575 (+ 144us) server_negotiation.cc:299] Negotiation successful
0929 10:49:59.870703 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":76,"threads_started":1}
I20260929 10:49:59.876519 21423 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:49:59.877254 21423 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:49:59.886128 21423 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f421038f9d7049d6806440dabf875ab8: Generated new TSK 0
I20260929 10:49:59.886317 21423 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.80.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
I20260929 10:49:59.920606 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.193:0
--local_ip_for_outbound_sockets=127.20.80.193
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37351
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:33669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:00.018958 21435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:00.019200 21435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:00.019241 21435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:00.020471 21435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:00.020539 21435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:00.020589 21435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.193
I20260929 10:50:00.022221 21435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:33669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37351
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.193
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:00.022511 21435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:00.022753 21435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:00.027992 21441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:00.027985 21443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:00.028076 21435 server_base.cc:1034] running on GCE node
W20260929 10:50:00.027982 21440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:00.028415 21435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:00.028682 21435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:00.029868 21435 hybrid_clock.cc:648] HybridClock initialized: now 1790679000029857 us; error 33 us; skew 500 ppm
Sep 29 10:50:00 dist-test-slave-vqch krb5kdc[21328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679000, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:00.032519 21435 init.cc:377] Logged in from keytab as kudu/127.20.80.193@KRBTEST.COM (short username kudu)
I20260929 10:50:00.032968 21435 webserver.cc:460] Webserver started at http://127.20.80.193:32989/ using document root <none> and password file <none>
I20260929 10:50:00.033272 21435 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:00.033339 21435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:00.033488 21435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:00.034622 21435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-0/data/instance:
uuid: "4f724c674b5b43f58b99e798b1130693"
format_stamp: "Formatted at 2026-09-29 10:50:00 on dist-test-slave-vqch"
server_key: "7335ce0c74683ec28c08628a637e78e4"
server_key_iv: "fd294617716044db5cd64203c18fff04"
server_key_version: "encryptionkey@0"
I20260929 10:50:00.035006 21435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance:
uuid: "4f724c674b5b43f58b99e798b1130693"
format_stamp: "Formatted at 2026-09-29 10:50:00 on dist-test-slave-vqch"
server_key: "7335ce0c74683ec28c08628a637e78e4"
server_key_iv: "fd294617716044db5cd64203c18fff04"
server_key_version: "encryptionkey@0"
I20260929 10:50:00.036376 21435 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 10:50:00.037171 21450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:00.037413 21435 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:50:00.037534 21435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-0/wal
uuid: "4f724c674b5b43f58b99e798b1130693"
format_stamp: "Formatted at 2026-09-29 10:50:00 on dist-test-slave-vqch"
server_key: "7335ce0c74683ec28c08628a637e78e4"
server_key_iv: "fd294617716044db5cd64203c18fff04"
server_key_version: "encryptionkey@0"
I20260929 10:50:00.037623 21435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:00.051252 21435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:00.053373 21435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:00.053548 21435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:00.053802 21435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:00.054132 21435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:00.054199 21435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:00.054245 21435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:00.054275 21435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:00.060642 21435 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.193:40123
I20260929 10:50:00.060711 21563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.193:40123 every 8 connection(s)
I20260929 10:50:00.061088 21435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
Sep 29 10:50:00 dist-test-slave-vqch krb5kdc[21328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679000, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:00.068878 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 21435
I20260929 10:50:00.068984 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance
I20260929 10:50:00.069185 20803 external_mini_cluster.cc:1428] Setting key 591fe4265e4214e8a62248a0495452ce
I20260929 10:50:00.070716 21413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:00.061820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:48175 (local address 127.20.80.254:37351)
0929 10:50:00.061893 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:00.061895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:00.062093 (+ 198us) server_negotiation.cc:407] Connection header received
0929 10:50:00.063014 (+ 921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:00.063015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:00.063025 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:00.063091 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:00.063662 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.064288 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.065262 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.065563 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.065625 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:00.067680 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:00.067681 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:00.067682 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:00.067688 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:00.069056 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:00.069559 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:00.069560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:00.069560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:00.069631 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:00.070000 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:00.070001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:00.070001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:00.070140 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:00.070205 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:00.070556 (+ 351us) server_negotiation.cc:299] Negotiation successful
0929 10:50:00.070643 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 10:50:00.070982 21566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:00.061798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37351 (local address 127.20.80.193:48175)
0929 10:50:00.061965 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:00.062007 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:00.062901 (+ 894us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:00.063149 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:00.063153 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:00.063403 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:00.063590 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:00.063594 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.064358 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.064359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:00.065187 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:00.065189 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.065639 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.065640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:00.065739 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:00.066287 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:00.066296 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:00.067598 (+ 1302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:00.069155 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:00.069163 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:00.069165 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:00.069439 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:00.069746 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:00.069747 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:00.069747 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:00.069917 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:00.070233 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:00.070241 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:00.070406 (+ 165us) client_negotiation.cc:769] Sending connection context
0929 10:50:00.070532 (+ 126us) client_negotiation.cc:240] Negotiation successful
0929 10:50:00.070707 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":97,"threads_started":1}
I20260929 10:50:00.071969 21564 heartbeater.cc:347] Connected to a master server at 127.20.80.254:37351
I20260929 10:50:00.072060 21564 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:00.072252 21564 heartbeater.cc:511] Master 127.20.80.254:37351 requested a full tablet report, sending...
I20260929 10:50:00.072885 21373 ts_manager.cc:194] Registered new tserver with Master: 4f724c674b5b43f58b99e798b1130693 (127.20.80.193:40123)
I20260929 10:50:00.073843 21373 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.193@KRBTEST.COM'} at 127.20.80.193:48175
WARNING: no policy specified for kudu/127.20.80.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
I20260929 10:50:00.097124 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.194:0
--local_ip_for_outbound_sockets=127.20.80.194
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37351
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:33669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:00.179076 21569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:00.179279 21569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:00.179303 21569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:00.180390 21569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:00.180425 21569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:00.180451 21569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.194
I20260929 10:50:00.181999 21569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:33669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37351
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.194
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:00.182191 21569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:00.182363 21569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:00.187237 21577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:00.187316 21575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:00.187395 21569 server_base.cc:1034] running on GCE node
W20260929 10:50:00.187270 21574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:00.187688 21569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:00.187891 21569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:00.189061 21569 hybrid_clock.cc:648] HybridClock initialized: now 1790679000189050 us; error 35 us; skew 500 ppm
Sep 29 10:50:00 dist-test-slave-vqch krb5kdc[21328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679000, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:00.191601 21569 init.cc:377] Logged in from keytab as kudu/127.20.80.194@KRBTEST.COM (short username kudu)
I20260929 10:50:00.192035 21569 webserver.cc:460] Webserver started at http://127.20.80.194:34649/ using document root <none> and password file <none>
I20260929 10:50:00.192265 21569 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:00.192333 21569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:00.192505 21569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:00.193622 21569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-1/data/instance:
uuid: "80e4a77f6c4449c5b51e599c4431f8e6"
format_stamp: "Formatted at 2026-09-29 10:50:00 on dist-test-slave-vqch"
server_key: "eee80dbd8b6ca3e92e0194b4d2084ec4"
server_key_iv: "aef78bb6602bf1e9eb143b2af3ae994e"
server_key_version: "encryptionkey@0"
I20260929 10:50:00.193995 21569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance:
uuid: "80e4a77f6c4449c5b51e599c4431f8e6"
format_stamp: "Formatted at 2026-09-29 10:50:00 on dist-test-slave-vqch"
server_key: "eee80dbd8b6ca3e92e0194b4d2084ec4"
server_key_iv: "aef78bb6602bf1e9eb143b2af3ae994e"
server_key_version: "encryptionkey@0"
I20260929 10:50:00.195298 21569 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:50:00.196081 21584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:00.196264 21569 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:00.196369 21569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-1/wal
uuid: "80e4a77f6c4449c5b51e599c4431f8e6"
format_stamp: "Formatted at 2026-09-29 10:50:00 on dist-test-slave-vqch"
server_key: "eee80dbd8b6ca3e92e0194b4d2084ec4"
server_key_iv: "aef78bb6602bf1e9eb143b2af3ae994e"
server_key_version: "encryptionkey@0"
I20260929 10:50:00.196453 21569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:00.226090 21569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:00.228264 21569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:00.228426 21569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:00.228690 21569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:00.229061 21569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:00.229115 21569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:00.229156 21569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:00.229199 21569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:00.235152 21569 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.194:41781
I20260929 10:50:00.235203 21697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.194:41781 every 8 connection(s)
I20260929 10:50:00.235514 21569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
Sep 29 10:50:00 dist-test-slave-vqch krb5kdc[21328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679000, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:00.242406 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 21569
I20260929 10:50:00.242477 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance
I20260929 10:50:00.242645 20803 external_mini_cluster.cc:1428] Setting key c4c22797a14689c3042bbe9ef82264ee
I20260929 10:50:00.245345 21413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:00.236271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:46029 (local address 127.20.80.254:37351)
0929 10:50:00.236333 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:00.236336 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:00.236553 (+ 217us) server_negotiation.cc:407] Connection header received
0929 10:50:00.237609 (+ 1056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:00.237610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:00.237621 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:00.237674 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:00.238285 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.238796 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.239752 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.239948 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.239998 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:00.242152 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:00.242153 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:00.242154 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:00.242160 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:00.243938 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:00.244302 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:00.244303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:00.244303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:00.244369 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:00.244590 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:00.244591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:00.244591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:00.244746 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:00.244822 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:00.245196 (+ 374us) server_negotiation.cc:299] Negotiation successful
0929 10:50:00.245280 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 10:50:00.245589 21700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:00.236258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37351 (local address 127.20.80.194:46029)
0929 10:50:00.236429 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:00.236464 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:00.237517 (+ 1053us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:00.237748 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:00.237751 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:00.237998 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:00.238210 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:00.238214 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.238874 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.238874 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:00.239677 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:00.239679 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.240017 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.240018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:00.240114 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:00.240622 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:00.240631 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:00.242064 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:00.244014 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:00.244020 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:00.244022 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:00.244228 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:00.244435 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:00.244436 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:00.244436 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:00.244527 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:00.244823 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:00.244830 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:00.245021 (+ 191us) client_negotiation.cc:769] Sending connection context
0929 10:50:00.245145 (+ 124us) client_negotiation.cc:240] Negotiation successful
0929 10:50:00.245335 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":69,"threads_started":1}
I20260929 10:50:00.246037 21698 heartbeater.cc:347] Connected to a master server at 127.20.80.254:37351
I20260929 10:50:00.246141 21698 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:00.246331 21698 heartbeater.cc:511] Master 127.20.80.254:37351 requested a full tablet report, sending...
I20260929 10:50:00.246709 21373 ts_manager.cc:194] Registered new tserver with Master: 80e4a77f6c4449c5b51e599c4431f8e6 (127.20.80.194:41781)
I20260929 10:50:00.247264 21373 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.194@KRBTEST.COM'} at 127.20.80.194:46029
WARNING: no policy specified for kudu/127.20.80.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
I20260929 10:50:00.269613 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.195:0
--local_ip_for_outbound_sockets=127.20.80.195
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37351
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:33669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:00.350589 21703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:00.350827 21703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:00.350865 21703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:00.352041 21703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:00.352104 21703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:00.352151 21703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.195
I20260929 10:50:00.353721 21703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:33669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37351
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.195
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:00.353997 21703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:00.354228 21703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:00.359302 21709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:00.359319 21708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:00.359318 21703 server_base.cc:1034] running on GCE node
W20260929 10:50:00.359302 21711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:00.359748 21703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:00.359994 21703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:00.361166 21703 hybrid_clock.cc:648] HybridClock initialized: now 1790679000361155 us; error 44 us; skew 500 ppm
Sep 29 10:50:00 dist-test-slave-vqch krb5kdc[21328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679000, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:00.363539 21703 init.cc:377] Logged in from keytab as kudu/127.20.80.195@KRBTEST.COM (short username kudu)
I20260929 10:50:00.364039 21703 webserver.cc:460] Webserver started at http://127.20.80.195:36047/ using document root <none> and password file <none>
I20260929 10:50:00.364286 21703 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:00.364359 21703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:00.364506 21703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:00.365725 21703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-2/data/instance:
uuid: "c98541e358964d64a70c5bcddf82a610"
format_stamp: "Formatted at 2026-09-29 10:50:00 on dist-test-slave-vqch"
server_key: "a234926c22f5f7532f16b0aeccf5b668"
server_key_iv: "25db441e2b2d5aaa1b0cd9a2ca6746c6"
server_key_version: "encryptionkey@0"
I20260929 10:50:00.366086 21703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance:
uuid: "c98541e358964d64a70c5bcddf82a610"
format_stamp: "Formatted at 2026-09-29 10:50:00 on dist-test-slave-vqch"
server_key: "a234926c22f5f7532f16b0aeccf5b668"
server_key_iv: "25db441e2b2d5aaa1b0cd9a2ca6746c6"
server_key_version: "encryptionkey@0"
I20260929 10:50:00.367427 21703 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:50:00.368140 21718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:00.368328 21703 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:00.368418 21703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-2/wal
uuid: "c98541e358964d64a70c5bcddf82a610"
format_stamp: "Formatted at 2026-09-29 10:50:00 on dist-test-slave-vqch"
server_key: "a234926c22f5f7532f16b0aeccf5b668"
server_key_iv: "25db441e2b2d5aaa1b0cd9a2ca6746c6"
server_key_version: "encryptionkey@0"
I20260929 10:50:00.368492 21703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:00.381356 21703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:00.383174 21703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:00.383342 21703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:00.383585 21703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:00.383904 21703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:00.383963 21703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:00.383999 21703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:00.384018 21703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:00.389976 21703 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.195:33099
I20260929 10:50:00.390033 21831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.195:33099 every 8 connection(s)
I20260929 10:50:00.390360 21703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
I20260929 10:50:00.394260 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 21703
I20260929 10:50:00.394351 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance
I20260929 10:50:00.394490 20803 external_mini_cluster.cc:1428] Setting key 881eb84608dfdd79053c9a84e6df9c42
Sep 29 10:50:00 dist-test-slave-vqch krb5kdc[21328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679000, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:00.399643 21413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:00.391057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:49409 (local address 127.20.80.254:37351)
0929 10:50:00.391100 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:00.391102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:00.391321 (+ 219us) server_negotiation.cc:407] Connection header received
0929 10:50:00.392219 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:00.392220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:00.392230 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:00.392333 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:00.392903 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.393419 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.394717 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.394971 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.395062 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:00.397261 (+ 2199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:00.397263 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:00.397263 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:00.397269 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:00.398421 (+ 1152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:00.398768 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:00.398769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:00.398784 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:00.398835 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:00.399122 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:00.399122 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:00.399123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:00.399223 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:00.399277 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:00.399518 (+ 241us) server_negotiation.cc:299] Negotiation successful
0929 10:50:00.399579 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 10:50:00.399909 21834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:00.391053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37351 (local address 127.20.80.195:49409)
0929 10:50:00.391207 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:00.391240 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:00.392121 (+ 881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:00.392398 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:00.392401 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:00.392643 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:00.392831 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:00.392835 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.393525 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.393527 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:00.394581 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:00.394584 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.395072 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.395073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:00.395200 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:00.395742 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:00.395750 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:00.397181 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:00.398487 (+ 1306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:00.398493 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:00.398495 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:00.398705 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:00.398919 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:00.398920 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:00.398920 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:00.399037 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:00.399290 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:00.399298 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:00.399407 (+ 109us) client_negotiation.cc:769] Sending connection context
0929 10:50:00.399489 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 10:50:00.399624 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":91,"threads_started":1}
I20260929 10:50:00.400440 21832 heartbeater.cc:347] Connected to a master server at 127.20.80.254:37351
I20260929 10:50:00.400533 21832 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:00.400672 21832 heartbeater.cc:511] Master 127.20.80.254:37351 requested a full tablet report, sending...
I20260929 10:50:00.401125 21373 ts_manager.cc:194] Registered new tserver with Master: c98541e358964d64a70c5bcddf82a610 (127.20.80.195:33099)
I20260929 10:50:00.401463 21373 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.195@KRBTEST.COM'} at 127.20.80.195:49409
I20260929 10:50:00.406114 20803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:50:00.413364 21413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:00.407639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60622 (local address 127.20.80.254:37351)
0929 10:50:00.407708 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:00.407710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:00.407797 (+ 87us) server_negotiation.cc:407] Connection header received
0929 10:50:00.407863 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:00.407863 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:00.407873 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:00.407927 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:00.408465 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.408939 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.409683 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.409880 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.409950 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:00.411009 (+ 1059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:00.411011 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:00.411011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:00.411018 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:00.412188 (+ 1170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:00.412555 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:00.412556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:00.412556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:00.412612 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:00.412809 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:00.412810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:00.412810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:00.412954 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:00.413079 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:00.413185 (+ 106us) server_negotiation.cc:299] Negotiation successful
0929 10:50:00.413264 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 10:50:00.415447 21373 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60622:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:50:00.417040 21373 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:50:00.423857 21843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:00.419323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:40123 (local address 127.0.0.1:49194)
0929 10:50:00.419671 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:00.419702 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:00.419754 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:00.420042 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:00.420045 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:00.420047 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:00.420099 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:00.420100 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.421365 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.421367 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:00.422907 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:00.422909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.423561 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.423563 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:00.423643 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:00.423656 (+ 13us) client_negotiation.cc:769] Sending connection context
0929 10:50:00.423722 (+ 66us) client_negotiation.cc:240] Negotiation successful
0929 10:50:00.423783 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"mutex_wait_us":5,"thread_start_us":56,"threads_started":1}
I20260929 10:50:00.423864 21845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:00.419464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49194 (local address 127.20.80.193:40123)
0929 10:50:00.419891 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:00.419895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:00.419912 (+ 17us) server_negotiation.cc:407] Connection header received
0929 10:50:00.419945 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:00.419947 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:00.419959 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:00.420020 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:00.420279 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.421272 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.422966 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.423461 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.423559 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:00.423620 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:00.423743 (+ 123us) server_negotiation.cc:299] Negotiation successful
0929 10:50:00.423784 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":89,"threads_started":1}
I20260929 10:50:00.423983 21847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:00.419531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:33099 (local address 127.0.0.1:54506)
0929 10:50:00.420411 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:00.420421 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:00.420474 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:00.420630 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:00.420631 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:00.420633 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:00.420688 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:00.420689 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.421707 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.421708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:00.422807 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:00.422809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.423743 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.423745 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:00.423809 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:00.423814 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 10:50:00.423862 (+ 48us) client_negotiation.cc:240] Negotiation successful
0929 10:50:00.423903 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":863,"thread_start_us":74,"threads_started":1}
I20260929 10:50:00.424209 21848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:00.419642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54506 (local address 127.20.80.195:33099)
0929 10:50:00.419862 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:00.419867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:00.420469 (+ 602us) server_negotiation.cc:407] Connection header received
0929 10:50:00.420534 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:00.420536 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:00.420550 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:00.420610 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:00.420750 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.421585 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.422891 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.423584 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.423651 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:00.423709 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:00.424096 (+ 387us) server_negotiation.cc:299] Negotiation successful
0929 10:50:00.424139 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":122,"threads_started":1}
I20260929 10:50:00.424897 21844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:00.419324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:41781 (local address 127.0.0.1:46542)
0929 10:50:00.419660 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:00.419701 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:00.419764 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:00.421367 (+ 1603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:00.421368 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:00.421370 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:00.421401 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:00.421402 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.422698 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.422699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:00.423764 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:00.423766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.424655 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.424656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:00.424731 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:00.424735 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:50:00.424791 (+ 56us) client_negotiation.cc:240] Negotiation successful
0929 10:50:00.424831 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"spinlock_wait_cycles":384,"thread_start_us":82,"threads_started":1}
I20260929 10:50:00.424909 21846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:00.419507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46542 (local address 127.20.80.194:41781)
0929 10:50:00.420139 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:00.420142 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:00.420149 (+ 7us) server_negotiation.cc:407] Connection header received
0929 10:50:00.420217 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:00.420219 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:00.420235 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:00.420307 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:00.421475 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.422583 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.423848 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.424573 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.424652 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:00.424703 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:00.424804 (+ 101us) server_negotiation.cc:299] Negotiation successful
0929 10:50:00.424839 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":84,"threads_started":1}
I20260929 10:50:00.425077 21766 tablet_service.cc:1467] Processing CreateTablet for tablet e118a56b6727450797fb19bd781a727d (DEFAULT_TABLE table=test-table [id=a340e79b0ee446c3bf56b7320041ccdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:00.425468 21766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e118a56b6727450797fb19bd781a727d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:00.428347 21849 tablet_bootstrap.cc:492] T e118a56b6727450797fb19bd781a727d P c98541e358964d64a70c5bcddf82a610: Bootstrap starting.
I20260929 10:50:00.428669 21498 tablet_service.cc:1467] Processing CreateTablet for tablet e118a56b6727450797fb19bd781a727d (DEFAULT_TABLE table=test-table [id=a340e79b0ee446c3bf56b7320041ccdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:00.429064 21498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e118a56b6727450797fb19bd781a727d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:00.429122 21849 tablet_bootstrap.cc:654] T e118a56b6727450797fb19bd781a727d P c98541e358964d64a70c5bcddf82a610: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:00.429186 21632 tablet_service.cc:1467] Processing CreateTablet for tablet e118a56b6727450797fb19bd781a727d (DEFAULT_TABLE table=test-table [id=a340e79b0ee446c3bf56b7320041ccdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:00.429538 21632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e118a56b6727450797fb19bd781a727d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:00.429725 21849 log.cc:824] T e118a56b6727450797fb19bd781a727d P c98541e358964d64a70c5bcddf82a610: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:00.430554 21849 tablet_bootstrap.cc:492] T e118a56b6727450797fb19bd781a727d P c98541e358964d64a70c5bcddf82a610: No bootstrap required, opened a new log
I20260929 10:50:00.430629 21849 ts_tablet_manager.cc:1397] T e118a56b6727450797fb19bd781a727d P c98541e358964d64a70c5bcddf82a610: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260929 10:50:00.431939 21851 tablet_bootstrap.cc:492] T e118a56b6727450797fb19bd781a727d P 4f724c674b5b43f58b99e798b1130693: Bootstrap starting.
I20260929 10:50:00.432530 21851 tablet_bootstrap.cc:654] T e118a56b6727450797fb19bd781a727d P 4f724c674b5b43f58b99e798b1130693: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:00.432866 21852 tablet_bootstrap.cc:492] T e118a56b6727450797fb19bd781a727d P 80e4a77f6c4449c5b51e599c4431f8e6: Bootstrap starting.
I20260929 10:50:00.432766 21849 raft_consensus.cc:348] T e118a56b6727450797fb19bd781a727d P c98541e358964d64a70c5bcddf82a610 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f724c674b5b43f58b99e798b1130693" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 40123 } } peers { permanent_uuid: "80e4a77f6c4449c5b51e599c4431f8e6" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 41781 } } peers { permanent_uuid: "c98541e358964d64a70c5bcddf82a610" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 33099 } }
I20260929 10:50:00.432934 21849 raft_consensus.cc:374] T e118a56b6727450797fb19bd781a727d P c98541e358964d64a70c5bcddf82a610 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:00.433012 21849 raft_consensus.cc:729] T e118a56b6727450797fb19bd781a727d P c98541e358964d64a70c5bcddf82a610 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c98541e358964d64a70c5bcddf82a610, State: Initialized, Role: FOLLOWER
I20260929 10:50:00.433135 21849 consensus_queue.cc:260] T e118a56b6727450797fb19bd781a727d P c98541e358964d64a70c5bcddf82a610 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f724c674b5b43f58b99e798b1130693" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 40123 } } peers { permanent_uuid: "80e4a77f6c4449c5b51e599c4431f8e6" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 41781 } } peers { permanent_uuid: "c98541e358964d64a70c5bcddf82a610" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 33099 } }
I20260929 10:50:00.433262 21851 log.cc:824] T e118a56b6727450797fb19bd781a727d P 4f724c674b5b43f58b99e798b1130693: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:00.433432 21849 ts_tablet_manager.cc:1428] T e118a56b6727450797fb19bd781a727d P c98541e358964d64a70c5bcddf82a610: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260929 10:50:00.433580 21832 heartbeater.cc:503] Master 127.20.80.254:37351 was elected leader, sending a full tablet report...
I20260929 10:50:00.433629 21852 tablet_bootstrap.cc:654] T e118a56b6727450797fb19bd781a727d P 80e4a77f6c4449c5b51e599c4431f8e6: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:00.434118 21851 tablet_bootstrap.cc:492] T e118a56b6727450797fb19bd781a727d P 4f724c674b5b43f58b99e798b1130693: No bootstrap required, opened a new log
I20260929 10:50:00.434165 21852 log.cc:824] T e118a56b6727450797fb19bd781a727d P 80e4a77f6c4449c5b51e599c4431f8e6: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:00.434199 21851 ts_tablet_manager.cc:1397] T e118a56b6727450797fb19bd781a727d P 4f724c674b5b43f58b99e798b1130693: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260929 10:50:00.434963 21852 tablet_bootstrap.cc:492] T e118a56b6727450797fb19bd781a727d P 80e4a77f6c4449c5b51e599c4431f8e6: No bootstrap required, opened a new log
I20260929 10:50:00.435036 21852 ts_tablet_manager.cc:1397] T e118a56b6727450797fb19bd781a727d P 80e4a77f6c4449c5b51e599c4431f8e6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:50:00.436334 21851 raft_consensus.cc:348] T e118a56b6727450797fb19bd781a727d P 4f724c674b5b43f58b99e798b1130693 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f724c674b5b43f58b99e798b1130693" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 40123 } } peers { permanent_uuid: "80e4a77f6c4449c5b51e599c4431f8e6" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 41781 } } peers { permanent_uuid: "c98541e358964d64a70c5bcddf82a610" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 33099 } }
I20260929 10:50:00.436468 21852 raft_consensus.cc:348] T e118a56b6727450797fb19bd781a727d P 80e4a77f6c4449c5b51e599c4431f8e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f724c674b5b43f58b99e798b1130693" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 40123 } } peers { permanent_uuid: "80e4a77f6c4449c5b51e599c4431f8e6" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 41781 } } peers { permanent_uuid: "c98541e358964d64a70c5bcddf82a610" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 33099 } }
I20260929 10:50:00.436573 21851 raft_consensus.cc:374] T e118a56b6727450797fb19bd781a727d P 4f724c674b5b43f58b99e798b1130693 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:00.436591 21852 raft_consensus.cc:374] T e118a56b6727450797fb19bd781a727d P 80e4a77f6c4449c5b51e599c4431f8e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:00.436625 21851 raft_consensus.cc:729] T e118a56b6727450797fb19bd781a727d P 4f724c674b5b43f58b99e798b1130693 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f724c674b5b43f58b99e798b1130693, State: Initialized, Role: FOLLOWER
I20260929 10:50:00.436661 21852 raft_consensus.cc:729] T e118a56b6727450797fb19bd781a727d P 80e4a77f6c4449c5b51e599c4431f8e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80e4a77f6c4449c5b51e599c4431f8e6, State: Initialized, Role: FOLLOWER
I20260929 10:50:00.436743 21851 consensus_queue.cc:260] T e118a56b6727450797fb19bd781a727d P 4f724c674b5b43f58b99e798b1130693 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f724c674b5b43f58b99e798b1130693" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 40123 } } peers { permanent_uuid: "80e4a77f6c4449c5b51e599c4431f8e6" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 41781 } } peers { permanent_uuid: "c98541e358964d64a70c5bcddf82a610" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 33099 } }
I20260929 10:50:00.436775 21852 consensus_queue.cc:260] T e118a56b6727450797fb19bd781a727d P 80e4a77f6c4449c5b51e599c4431f8e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f724c674b5b43f58b99e798b1130693" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 40123 } } peers { permanent_uuid: "80e4a77f6c4449c5b51e599c4431f8e6" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 41781 } } peers { permanent_uuid: "c98541e358964d64a70c5bcddf82a610" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 33099 } }
I20260929 10:50:00.437065 21852 ts_tablet_manager.cc:1428] T e118a56b6727450797fb19bd781a727d P 80e4a77f6c4449c5b51e599c4431f8e6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:50:00.437109 21564 heartbeater.cc:503] Master 127.20.80.254:37351 was elected leader, sending a full tablet report...
I20260929 10:50:00.437078 21698 heartbeater.cc:503] Master 127.20.80.254:37351 was elected leader, sending a full tablet report...
I20260929 10:50:00.437286 21851 ts_tablet_manager.cc:1428] T e118a56b6727450797fb19bd781a727d P 4f724c674b5b43f58b99e798b1130693: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
W20260929 10:50:00.486140 21699 tablet.cc:2369] T e118a56b6727450797fb19bd781a727d P 80e4a77f6c4449c5b51e599c4431f8e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:50:00.561928 21565 tablet.cc:2369] T e118a56b6727450797fb19bd781a727d P 4f724c674b5b43f58b99e798b1130693: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:50:00.641053 21833 tablet.cc:2369] T e118a56b6727450797fb19bd781a727d P c98541e358964d64a70c5bcddf82a610: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:50:00.692873 21857 raft_consensus.cc:482] T e118a56b6727450797fb19bd781a727d P 4f724c674b5b43f58b99e798b1130693 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:50:00.693060 21857 raft_consensus.cc:504] T e118a56b6727450797fb19bd781a727d P 4f724c674b5b43f58b99e798b1130693 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f724c674b5b43f58b99e798b1130693" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 40123 } } peers { permanent_uuid: "80e4a77f6c4449c5b51e599c4431f8e6" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 41781 } } peers { permanent_uuid: "c98541e358964d64a70c5bcddf82a610" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 33099 } }
I20260929 10:50:00.693455 21857 leader_election.cc:288] T e118a56b6727450797fb19bd781a727d P 4f724c674b5b43f58b99e798b1130693 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80e4a77f6c4449c5b51e599c4431f8e6 (127.20.80.194:41781), c98541e358964d64a70c5bcddf82a610 (127.20.80.195:33099)
I20260929 10:50:00.696659 21859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:00.693896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:33099 (local address 127.20.80.193:43547)
0929 10:50:00.694038 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:00.694048 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:00.694109 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:00.694309 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:00.694310 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:00.694312 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:00.694382 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:00.694384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.695062 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.695063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:00.695864 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:00.695866 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.696425 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.696426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:00.696501 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:00.696507 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 10:50:00.696562 (+ 55us) client_negotiation.cc:240] Negotiation successful
0929 10:50:00.696594 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":4,"thread_start_us":69,"threads_started":1}
I20260929 10:50:00.696753 21848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:00.693969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:43547 (local address 127.20.80.195:33099)
0929 10:50:00.694018 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:00.694020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:00.694110 (+ 90us) server_negotiation.cc:407] Connection header received
0929 10:50:00.694165 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:00.694166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:00.694175 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:00.694296 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:00.694487 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.694956 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.695959 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.696343 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.696401 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:00.696408 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:00.696604 (+ 196us) server_negotiation.cc:299] Negotiation successful
0929 10:50:00.696652 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260929 10:50:00.696907 21846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:00.693784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:60233 (local address 127.20.80.194:41781)
0929 10:50:00.693852 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:00.693854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:00.693908 (+ 54us) server_negotiation.cc:407] Connection header received
0929 10:50:00.694002 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:00.694004 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:00.694015 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:00.694087 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:00.694443 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.694931 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.696001 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.696404 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.696478 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:00.696487 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:00.696791 (+ 304us) server_negotiation.cc:299] Negotiation successful
0929 10:50:00.696846 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 10:50:00.697204 21858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:00.693671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:41781 (local address 127.20.80.193:60233)
0929 10:50:00.693826 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:00.693842 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:00.693930 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:00.694124 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:00.694125 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:00.694128 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:00.694348 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:00.694351 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.695013 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.695014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:00.695865 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:00.695868 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.696497 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.696498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:00.696686 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:00.696691 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 10:50:00.697041 (+ 350us) client_negotiation.cc:240] Negotiation successful
0929 10:50:00.697086 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":2,"thread_start_us":83,"threads_started":1}
I20260929 10:50:00.697213 21786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e118a56b6727450797fb19bd781a727d" candidate_uuid: "4f724c674b5b43f58b99e798b1130693" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c98541e358964d64a70c5bcddf82a610" is_pre_election: true
I20260929 10:50:00.697394 21786 raft_consensus.cc:2454] T e118a56b6727450797fb19bd781a727d P c98541e358964d64a70c5bcddf82a610 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f724c674b5b43f58b99e798b1130693 in term 0.
I20260929 10:50:00.697542 21652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e118a56b6727450797fb19bd781a727d" candidate_uuid: "4f724c674b5b43f58b99e798b1130693" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80e4a77f6c4449c5b51e599c4431f8e6" is_pre_election: true
I20260929 10:50:00.697613 21452 leader_election.cc:302] T e118a56b6727450797fb19bd781a727d P 4f724c674b5b43f58b99e798b1130693 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f724c674b5b43f58b99e798b1130693, c98541e358964d64a70c5bcddf82a610; no voters:
I20260929 10:50:00.697692 21652 raft_consensus.cc:2454] T e118a56b6727450797fb19bd781a727d P 80e4a77f6c4449c5b51e599c4431f8e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f724c674b5b43f58b99e798b1130693 in term 0.
I20260929 10:50:00.697775 21857 raft_consensus.cc:2781] T e118a56b6727450797fb19bd781a727d P 4f724c674b5b43f58b99e798b1130693 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:50:00.697839 21857 raft_consensus.cc:482] T e118a56b6727450797fb19bd781a727d P 4f724c674b5b43f58b99e798b1130693 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:50:00.697866 21857 raft_consensus.cc:3037] T e118a56b6727450797fb19bd781a727d P 4f724c674b5b43f58b99e798b1130693 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:00.698555 21857 raft_consensus.cc:504] T e118a56b6727450797fb19bd781a727d P 4f724c674b5b43f58b99e798b1130693 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f724c674b5b43f58b99e798b1130693" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 40123 } } peers { permanent_uuid: "80e4a77f6c4449c5b51e599c4431f8e6" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 41781 } } peers { permanent_uuid: "c98541e358964d64a70c5bcddf82a610" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 33099 } }
I20260929 10:50:00.698689 21857 leader_election.cc:288] T e118a56b6727450797fb19bd781a727d P 4f724c674b5b43f58b99e798b1130693 [CANDIDATE]: Term 1 election: Requested vote from peers 80e4a77f6c4449c5b51e599c4431f8e6 (127.20.80.194:41781), c98541e358964d64a70c5bcddf82a610 (127.20.80.195:33099)
I20260929 10:50:00.698891 21652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e118a56b6727450797fb19bd781a727d" candidate_uuid: "4f724c674b5b43f58b99e798b1130693" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80e4a77f6c4449c5b51e599c4431f8e6"
I20260929 10:50:00.698949 21786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e118a56b6727450797fb19bd781a727d" candidate_uuid: "4f724c674b5b43f58b99e798b1130693" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c98541e358964d64a70c5bcddf82a610"
I20260929 10:50:00.698973 21652 raft_consensus.cc:3037] T e118a56b6727450797fb19bd781a727d P 80e4a77f6c4449c5b51e599c4431f8e6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:00.699048 21786 raft_consensus.cc:3037] T e118a56b6727450797fb19bd781a727d P c98541e358964d64a70c5bcddf82a610 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:00.699702 21786 raft_consensus.cc:2454] T e118a56b6727450797fb19bd781a727d P c98541e358964d64a70c5bcddf82a610 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f724c674b5b43f58b99e798b1130693 in term 1.
I20260929 10:50:00.699815 21652 raft_consensus.cc:2454] T e118a56b6727450797fb19bd781a727d P 80e4a77f6c4449c5b51e599c4431f8e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f724c674b5b43f58b99e798b1130693 in term 1.
I20260929 10:50:00.699867 21452 leader_election.cc:302] T e118a56b6727450797fb19bd781a727d P 4f724c674b5b43f58b99e798b1130693 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f724c674b5b43f58b99e798b1130693, c98541e358964d64a70c5bcddf82a610; no voters:
I20260929 10:50:00.699975 21857 raft_consensus.cc:2781] T e118a56b6727450797fb19bd781a727d P 4f724c674b5b43f58b99e798b1130693 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:00.700213 21857 raft_consensus.cc:686] T e118a56b6727450797fb19bd781a727d P 4f724c674b5b43f58b99e798b1130693 [term 1 LEADER]: Becoming Leader. State: Replica: 4f724c674b5b43f58b99e798b1130693, State: Running, Role: LEADER
I20260929 10:50:00.700333 21857 consensus_queue.cc:237] T e118a56b6727450797fb19bd781a727d P 4f724c674b5b43f58b99e798b1130693 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f724c674b5b43f58b99e798b1130693" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 40123 } } peers { permanent_uuid: "80e4a77f6c4449c5b51e599c4431f8e6" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 41781 } } peers { permanent_uuid: "c98541e358964d64a70c5bcddf82a610" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 33099 } }
I20260929 10:50:00.701259 21373 catalog_manager.cc:5495] T e118a56b6727450797fb19bd781a727d P 4f724c674b5b43f58b99e798b1130693 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f724c674b5b43f58b99e798b1130693 (127.20.80.193). New cstate: current_term: 1 leader_uuid: "4f724c674b5b43f58b99e798b1130693" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f724c674b5b43f58b99e798b1130693" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 40123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "80e4a77f6c4449c5b51e599c4431f8e6" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 41781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c98541e358964d64a70c5bcddf82a610" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 33099 } health_report { overall_health: UNKNOWN } } }
Sep 29 10:50:00 dist-test-slave-vqch krb5kdc[21328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790678999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.193@KRBTEST.COM
I20260929 10:50:00.779871 21845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:00.772083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49200 (local address 127.20.80.193:40123)
0929 10:50:00.772143 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:00.772145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:00.772252 (+ 107us) server_negotiation.cc:407] Connection header received
0929 10:50:00.772359 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:00.772360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:00.772371 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:00.772441 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:00.773071 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.773571 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.774323 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.774514 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.774584 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:00.776959 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:00.777000 (+ 41us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:00.777001 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:00.777010 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:00.778587 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:00.779004 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:00.779007 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:00.779008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:00.779071 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:00.779290 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:00.779291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:00.779291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:00.779445 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:00.779636 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:00.779674 (+ 38us) server_negotiation.cc:299] Negotiation successful
0929 10:50:00.779762 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
Sep 29 10:50:00 dist-test-slave-vqch krb5kdc[21328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679000, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 10:50:00 dist-test-slave-vqch krb5kdc[21328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679000, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.80.193@KRBTEST.COM
I20260929 10:50:00.804747 21845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:00.796782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49214 (local address 127.20.80.193:40123)
0929 10:50:00.796878 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:00.796881 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:00.797032 (+ 151us) server_negotiation.cc:407] Connection header received
0929 10:50:00.797156 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:00.797157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:00.797169 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:00.797246 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:00.797996 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.798504 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.799294 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.799495 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.799587 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:00.802092 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:00.802093 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:00.802094 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:00.802101 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:00.803561 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:00.803928 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:00.803928 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:00.803929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:00.803983 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:00.804196 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:00.804197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:00.804198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:00.804311 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:00.804410 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:00.804584 (+ 174us) server_negotiation.cc:299] Negotiation successful
0929 10:50:00.804665 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Sep 29 10:50:00 dist-test-slave-vqch krb5kdc[21328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 10:50:00 dist-test-slave-vqch krb5kdc[21328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.193@KRBTEST.COM
I20260929 10:50:00.828981 21845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:00.821413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49222 (local address 127.20.80.193:40123)
0929 10:50:00.821512 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:00.821514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:00.821620 (+ 106us) server_negotiation.cc:407] Connection header received
0929 10:50:00.821646 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:00.821647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:00.821657 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:00.821700 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:00.822343 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.822826 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.823583 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:00.823813 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:00.823880 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:00.826329 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:00.826330 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:00.826331 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:00.826337 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:00.827836 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:00.828219 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:00.828220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:00.828220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:00.828272 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:00.828454 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:00.828455 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:00.828455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:00.828558 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:00.828667 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:00.828813 (+ 146us) server_negotiation.cc:299] Negotiation successful
0929 10:50:00.828875 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260929 10:50:00.829561 21478 tablet_service.cc:2908] Rejecting scan request for tablet e118a56b6727450797fb19bd781a727d: Uninitialized: clean time has not yet been initialized
I20260929 10:50:00.830483 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 21435
I20260929 10:50:00.836009 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 21569
I20260929 10:50:00.841290 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 21703
I20260929 10:50:00.846522 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 21341
2026-09-29T10:50:00Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2285 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:50:00 dist-test-slave-vqch krb5kdc[21874](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:50:00 dist-test-slave-vqch krb5kdc[21874](info): set up 2 sockets
Sep 29 10:50:00 dist-test-slave-vqch krb5kdc[21874](info): commencing operation
krb5kdc: starting...
W20260929 10:50:01.892306 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:50:01 dist-test-slave-vqch krb5kdc[21874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:50:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:50:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.80.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
I20260929 10:50:02.016418 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:42475
--webserver_interface=127.20.80.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:40303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:42475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:02.102885 21887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:02.103140 21887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:02.103180 21887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:02.104486 21887 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:50:02.104558 21887 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 10:50:02.104584 21887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:02.104622 21887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:02.104660 21887 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:50:02.104691 21887 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:50:02.106385 21887 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:40303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:42475
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:42475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:02.106693 21887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:02.106945 21887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:02.112291 21893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:02.112321 21895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:02.112293 21892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:02.112550 21887 server_base.cc:1034] running on GCE node
I20260929 10:50:02.112845 21887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:02.113198 21887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:02.114375 21887 hybrid_clock.cc:648] HybridClock initialized: now 1790679002114352 us; error 38 us; skew 500 ppm
Sep 29 10:50:02 dist-test-slave-vqch krb5kdc[21874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679002, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:02.122967 21887 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:50:02.123543 21887 webserver.cc:460] Webserver started at http://127.20.80.254:39783/ using document root <none> and password file <none>
I20260929 10:50:02.123795 21887 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:02.123869 21887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:02.124025 21887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:02.125281 21887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "2d9860e82bad4dda902f038e378b7410"
format_stamp: "Formatted at 2026-09-29 10:50:02 on dist-test-slave-vqch"
server_key: "0c2d7c35e37fdae97cb0444fbb33f786"
server_key_iv: "54cf83722e34a07bef3428b316a260fc"
server_key_version: "encryptionkey@0"
I20260929 10:50:02.125672 21887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "2d9860e82bad4dda902f038e378b7410"
format_stamp: "Formatted at 2026-09-29 10:50:02 on dist-test-slave-vqch"
server_key: "0c2d7c35e37fdae97cb0444fbb33f786"
server_key_iv: "54cf83722e34a07bef3428b316a260fc"
server_key_version: "encryptionkey@0"
I20260929 10:50:02.127183 21887 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:50:02.127977 21902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:02.128167 21887 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:02.128278 21887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "2d9860e82bad4dda902f038e378b7410"
format_stamp: "Formatted at 2026-09-29 10:50:02 on dist-test-slave-vqch"
server_key: "0c2d7c35e37fdae97cb0444fbb33f786"
server_key_iv: "54cf83722e34a07bef3428b316a260fc"
server_key_version: "encryptionkey@0"
I20260929 10:50:02.128361 21887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:02.140698 21887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:02.142980 21887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:02.143173 21887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:02.148311 21887 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:42475
I20260929 10:50:02.148367 21964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:42475 every 8 connection(s)
I20260929 10:50:02.148746 21887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:50:02.149437 21965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:02.151366 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 21887
I20260929 10:50:02.151487 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:50:02.151664 20803 external_mini_cluster.cc:1428] Setting key 2607561fc955f0c3569a6e659119ddac
I20260929 10:50:02.151716 21965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d9860e82bad4dda902f038e378b7410: Bootstrap starting.
I20260929 10:50:02.152485 21965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d9860e82bad4dda902f038e378b7410: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:02.152786 21965 log.cc:824] T 00000000000000000000000000000000 P 2d9860e82bad4dda902f038e378b7410: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:02.153797 21965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d9860e82bad4dda902f038e378b7410: No bootstrap required, opened a new log
I20260929 10:50:02.156365 21965 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d9860e82bad4dda902f038e378b7410 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d9860e82bad4dda902f038e378b7410" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 42475 } }
I20260929 10:50:02.156509 21965 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d9860e82bad4dda902f038e378b7410 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:02.156533 21965 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d9860e82bad4dda902f038e378b7410 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d9860e82bad4dda902f038e378b7410, State: Initialized, Role: FOLLOWER
I20260929 10:50:02.156705 21965 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d9860e82bad4dda902f038e378b7410 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d9860e82bad4dda902f038e378b7410" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 42475 } }
I20260929 10:50:02.156781 21965 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d9860e82bad4dda902f038e378b7410 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:50:02.156804 21965 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d9860e82bad4dda902f038e378b7410 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:50:02.156879 21965 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d9860e82bad4dda902f038e378b7410 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:02.157698 21965 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d9860e82bad4dda902f038e378b7410 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d9860e82bad4dda902f038e378b7410" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 42475 } }
I20260929 10:50:02.157824 21965 leader_election.cc:302] T 00000000000000000000000000000000 P 2d9860e82bad4dda902f038e378b7410 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d9860e82bad4dda902f038e378b7410; no voters:
I20260929 10:50:02.158006 21965 leader_election.cc:288] T 00000000000000000000000000000000 P 2d9860e82bad4dda902f038e378b7410 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:50:02.158066 21970 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d9860e82bad4dda902f038e378b7410 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:02.158259 21970 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d9860e82bad4dda902f038e378b7410 [term 1 LEADER]: Becoming Leader. State: Replica: 2d9860e82bad4dda902f038e378b7410, State: Running, Role: LEADER
I20260929 10:50:02.158308 21965 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d9860e82bad4dda902f038e378b7410 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:50:02.158368 21970 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d9860e82bad4dda902f038e378b7410 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d9860e82bad4dda902f038e378b7410" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 42475 } }
I20260929 10:50:02.158737 21972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d9860e82bad4dda902f038e378b7410 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d9860e82bad4dda902f038e378b7410" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d9860e82bad4dda902f038e378b7410" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 42475 } } }
I20260929 10:50:02.158726 21970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d9860e82bad4dda902f038e378b7410 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d9860e82bad4dda902f038e378b7410. Latest consensus state: current_term: 1 leader_uuid: "2d9860e82bad4dda902f038e378b7410" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d9860e82bad4dda902f038e378b7410" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 42475 } } }
I20260929 10:50:02.158890 21972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d9860e82bad4dda902f038e378b7410 [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:02.158906 21970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d9860e82bad4dda902f038e378b7410 [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:02.159245 21979 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 29 10:50:02 dist-test-slave-vqch krb5kdc[21874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:02.160025 21979 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:50:02.161603 21979 catalog_manager.cc:1347] Generated new cluster ID: 576ed4f43087438d801067e204e74f04
I20260929 10:50:02.161660 21979 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:50:02.163941 21968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:02.152192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37830 (local address 127.20.80.254:42475)
0929 10:50:02.152426 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:02.152435 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:02.152472 (+ 37us) server_negotiation.cc:407] Connection header received
0929 10:50:02.153084 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:02.153093 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:02.153159 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:02.153320 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:02.154208 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.155080 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.156251 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.156761 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.156894 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:02.160190 (+ 3296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:02.160194 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:02.160195 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:02.160210 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:02.162280 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:02.162692 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:02.162695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:02.162696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:02.162793 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:02.163029 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:02.163030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:02.163030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:02.163285 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:02.163466 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:02.163552 (+ 86us) server_negotiation.cc:299] Negotiation successful
0929 10:50:02.163718 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":90,"threads_started":1}
I20260929 10:50:02.187196 21979 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:50:02.187820 21979 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:50:02.194074 21979 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d9860e82bad4dda902f038e378b7410: Generated new TSK 0
I20260929 10:50:02.194278 21979 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.80.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
I20260929 10:50:02.228737 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.193:0
--local_ip_for_outbound_sockets=127.20.80.193
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:42475
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:40303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:02.325563 21991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:02.325765 21991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:02.325790 21991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:02.326946 21991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:02.326989 21991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:02.327010 21991 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 10:50:02.327028 21991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.193
I20260929 10:50:02.328488 21991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:40303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.80.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.80.254:42475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.193
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:02.328727 21991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:02.328972 21991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:02.334329 21999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:02.334476 21991 server_base.cc:1034] running on GCE node
W20260929 10:50:02.334328 21996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:02.334336 21997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:02.334805 21991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:02.335047 21991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:02.336221 21991 hybrid_clock.cc:648] HybridClock initialized: now 1790679002336211 us; error 43 us; skew 500 ppm
Sep 29 10:50:02 dist-test-slave-vqch krb5kdc[21874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679002, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:02.338768 21991 init.cc:377] Logged in from keytab as kudu/127.20.80.193@KRBTEST.COM (short username kudu)
I20260929 10:50:02.339319 21991 webserver.cc:460] Webserver started at http://127.20.80.193:35891/ using document root <none> and password file <none>
I20260929 10:50:02.339586 21991 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:02.339660 21991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:02.339857 21991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:02.341115 21991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-0/data/instance:
uuid: "a1d4c6dbe57b4d4a867fd490b4926816"
format_stamp: "Formatted at 2026-09-29 10:50:02 on dist-test-slave-vqch"
server_key: "fe432f54e9330d9874c607ba9fdc7965"
server_key_iv: "751588c756da99b4434b899e6dcf0dec"
server_key_version: "encryptionkey@0"
I20260929 10:50:02.341511 21991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance:
uuid: "a1d4c6dbe57b4d4a867fd490b4926816"
format_stamp: "Formatted at 2026-09-29 10:50:02 on dist-test-slave-vqch"
server_key: "fe432f54e9330d9874c607ba9fdc7965"
server_key_iv: "751588c756da99b4434b899e6dcf0dec"
server_key_version: "encryptionkey@0"
I20260929 10:50:02.343000 21991 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 10:50:02.343760 22006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:02.343952 21991 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:50:02.344049 21991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-0/wal
uuid: "a1d4c6dbe57b4d4a867fd490b4926816"
format_stamp: "Formatted at 2026-09-29 10:50:02 on dist-test-slave-vqch"
server_key: "fe432f54e9330d9874c607ba9fdc7965"
server_key_iv: "751588c756da99b4434b899e6dcf0dec"
server_key_version: "encryptionkey@0"
I20260929 10:50:02.344142 21991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:02.377704 21991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:02.379959 21991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:02.380158 21991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:02.380749 21991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:02.380820 21991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:02.380872 21991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:02.380901 21991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 10:50:02 dist-test-slave-vqch krb5kdc[21874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679002, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:02.388736 21991 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.193:34123
I20260929 10:50:02.388819 22121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.193:34123 every 8 connection(s)
I20260929 10:50:02.389196 21991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
I20260929 10:50:02.391535 21968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:02.381335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:52789 (local address 127.20.80.254:42475)
0929 10:50:02.381411 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:02.381413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:02.381672 (+ 259us) server_negotiation.cc:407] Connection header received
0929 10:50:02.382509 (+ 837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:02.382510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:02.382523 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:02.382585 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:02.383577 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.384470 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.385546 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.385777 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.385855 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:02.388591 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:02.388593 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:02.388593 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:02.388601 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:02.390227 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:02.390611 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:02.390612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:02.390612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:02.390713 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:02.390984 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:02.390984 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:02.390984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:02.391104 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:02.391163 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:02.391392 (+ 229us) server_negotiation.cc:299] Negotiation successful
0929 10:50:02.391466 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 10:50:02.391697 22015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:02.381309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:42475 (local address 127.20.80.193:52789)
0929 10:50:02.381505 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:02.381564 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:02.382357 (+ 793us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:02.382668 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:02.382684 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:02.383130 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:02.383468 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:02.383475 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.384566 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.384566 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:02.385448 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:02.385450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.385878 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.385879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:02.386026 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:02.386553 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:02.386563 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:02.388507 (+ 1944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:02.390306 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:02.390311 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:02.390313 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:02.390521 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:02.390806 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:02.390806 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:02.390807 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:02.390919 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:02.391163 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:02.391167 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:02.391291 (+ 124us) client_negotiation.cc:769] Sending connection context
0929 10:50:02.391369 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 10:50:02.391486 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":107,"threads_started":1}
I20260929 10:50:02.392248 22122 heartbeater.cc:347] Connected to a master server at 127.20.80.254:42475
I20260929 10:50:02.392333 22122 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:02.392539 22122 heartbeater.cc:511] Master 127.20.80.254:42475 requested a full tablet report, sending...
I20260929 10:50:02.393183 21918 ts_manager.cc:194] Registered new tserver with Master: a1d4c6dbe57b4d4a867fd490b4926816 (127.20.80.193:34123)
I20260929 10:50:02.394127 21918 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.193@KRBTEST.COM'} at 127.20.80.193:52789
I20260929 10:50:02.394696 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 21991
I20260929 10:50:02.394804 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance
I20260929 10:50:02.394953 20803 external_mini_cluster.cc:1428] Setting key d469057ec31927b25eec2d90b5f6534f
I20260929 10:50:02.399807 21968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:02.393469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:40447 (local address 127.20.80.254:42475)
0929 10:50:02.393512 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:02.393513 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:02.393595 (+ 82us) server_negotiation.cc:407] Connection header received
0929 10:50:02.393659 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:02.393660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:02.393668 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:02.393711 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:02.394024 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.394698 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.395603 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.395879 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.395947 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:02.396828 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:02.396829 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:02.396830 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:02.396837 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:02.398602 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:02.399021 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:02.399022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:02.399023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:02.399078 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:02.399285 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:02.399286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:02.399286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:02.399400 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:02.399476 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:02.399647 (+ 171us) server_negotiation.cc:299] Negotiation successful
0929 10:50:02.399719 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 10:50:02.399832 22128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:02.393387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:42475 (local address 127.20.80.193:40447)
0929 10:50:02.393536 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:02.393547 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:02.393607 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:02.393717 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:02.393718 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:02.393895 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:02.393965 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:02.393966 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.394797 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.394799 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:02.395511 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:02.395513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.395960 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.395961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:02.396017 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:02.396460 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:02.396464 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:02.396740 (+ 276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:02.398693 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:02.398694 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:02.398695 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:02.398925 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:02.399152 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:02.399153 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:02.399154 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:02.399212 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:02.399467 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:02.399469 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:02.399527 (+ 58us) client_negotiation.cc:769] Sending connection context
0929 10:50:02.399611 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 10:50:02.399719 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":98,"threads_started":1}
WARNING: no policy specified for kudu/127.20.80.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
I20260929 10:50:02.421780 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.194:0
--local_ip_for_outbound_sockets=127.20.80.194
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:42475
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:40303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:02.502277 22131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:02.502511 22131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:02.502552 22131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:02.503737 22131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:02.503800 22131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:02.503834 22131 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 10:50:02.503873 22131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.194
I20260929 10:50:02.505352 22131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:40303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.80.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.80.254:42475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.194
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:02.505607 22131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:02.505880 22131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:02.511426 22137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:02.511436 22139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:02.511428 22136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:02.511796 22131 server_base.cc:1034] running on GCE node
I20260929 10:50:02.512023 22131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:02.512329 22131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:02.513538 22131 hybrid_clock.cc:648] HybridClock initialized: now 1790679002513516 us; error 41 us; skew 500 ppm
Sep 29 10:50:02 dist-test-slave-vqch krb5kdc[21874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679002, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:02.516149 22131 init.cc:377] Logged in from keytab as kudu/127.20.80.194@KRBTEST.COM (short username kudu)
I20260929 10:50:02.516597 22131 webserver.cc:460] Webserver started at http://127.20.80.194:45229/ using document root <none> and password file <none>
I20260929 10:50:02.516790 22131 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:02.516831 22131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:02.516921 22131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:02.518051 22131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-1/data/instance:
uuid: "634f61a9482d490c9a3e58ac5690f289"
format_stamp: "Formatted at 2026-09-29 10:50:02 on dist-test-slave-vqch"
server_key: "569cec92aac7ed45616a8e000d71ad27"
server_key_iv: "4c3457d28b3ced0d1807e42a6cff3be8"
server_key_version: "encryptionkey@0"
I20260929 10:50:02.518357 22131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance:
uuid: "634f61a9482d490c9a3e58ac5690f289"
format_stamp: "Formatted at 2026-09-29 10:50:02 on dist-test-slave-vqch"
server_key: "569cec92aac7ed45616a8e000d71ad27"
server_key_iv: "4c3457d28b3ced0d1807e42a6cff3be8"
server_key_version: "encryptionkey@0"
I20260929 10:50:02.519586 22131 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:50:02.520309 22146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:02.520512 22131 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:02.520596 22131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-1/wal
uuid: "634f61a9482d490c9a3e58ac5690f289"
format_stamp: "Formatted at 2026-09-29 10:50:02 on dist-test-slave-vqch"
server_key: "569cec92aac7ed45616a8e000d71ad27"
server_key_iv: "4c3457d28b3ced0d1807e42a6cff3be8"
server_key_version: "encryptionkey@0"
I20260929 10:50:02.520654 22131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:02.534940 22131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:02.537302 22131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:02.537410 22131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:02.537855 22131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:02.537925 22131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:02.537973 22131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:02.537998 22131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:02.544551 22131 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.194:34501
I20260929 10:50:02.544621 22261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.194:34501 every 8 connection(s)
I20260929 10:50:02.544941 22131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
Sep 29 10:50:02 dist-test-slave-vqch krb5kdc[21874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679002, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:02.546491 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 22131
I20260929 10:50:02.546589 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance
I20260929 10:50:02.546792 20803 external_mini_cluster.cc:1428] Setting key 7cb6c6b880edc76f4b40a42a275b870d
I20260929 10:50:02.548707 21968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:02.538329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:54795 (local address 127.20.80.254:42475)
0929 10:50:02.538400 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:02.538403 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:02.538618 (+ 215us) server_negotiation.cc:407] Connection header received
0929 10:50:02.539114 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:02.539115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:02.539128 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:02.539206 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:02.540100 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.540949 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.542563 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.542906 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.542991 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:02.545610 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:02.545611 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:02.545611 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:02.545617 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:02.547186 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:02.547691 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:02.547692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:02.547693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:02.547752 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:02.548022 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:02.548023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:02.548024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:02.548143 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:02.548270 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:02.548538 (+ 268us) server_negotiation.cc:299] Negotiation successful
0929 10:50:02.548616 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 10:50:02.548967 22155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:02.538311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:42475 (local address 127.20.80.194:54795)
0929 10:50:02.538487 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:02.538525 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:02.539008 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:02.539292 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:02.539295 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:02.539698 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:02.539992 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:02.540000 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.541079 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.541081 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:02.542468 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:02.542472 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.542994 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.542995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:02.543094 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:02.543636 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:02.543648 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:02.545526 (+ 1878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:02.547276 (+ 1750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:02.547281 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:02.547283 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:02.547602 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:02.547828 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:02.547829 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:02.547829 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:02.547955 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:02.548262 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:02.548268 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:02.548405 (+ 137us) client_negotiation.cc:769] Sending connection context
0929 10:50:02.548515 (+ 110us) client_negotiation.cc:240] Negotiation successful
0929 10:50:02.548673 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":107,"threads_started":1}
I20260929 10:50:02.549482 22262 heartbeater.cc:347] Connected to a master server at 127.20.80.254:42475
I20260929 10:50:02.549580 22262 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:02.549789 22262 heartbeater.cc:511] Master 127.20.80.254:42475 requested a full tablet report, sending...
I20260929 10:50:02.550180 21918 ts_manager.cc:194] Registered new tserver with Master: 634f61a9482d490c9a3e58ac5690f289 (127.20.80.194:34501)
I20260929 10:50:02.550693 21918 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.194@KRBTEST.COM'} at 127.20.80.194:54795
WARNING: no policy specified for kudu/127.20.80.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.195@KRBTEST.COM" created.
I20260929 10:50:02.559499 21968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:02.553274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:59691 (local address 127.20.80.254:42475)
0929 10:50:02.553313 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:02.553315 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:02.553406 (+ 91us) server_negotiation.cc:407] Connection header received
0929 10:50:02.553467 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:02.553468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:02.553475 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:02.553516 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:02.553883 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.554340 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.555392 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.555610 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.555665 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:02.556557 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:02.556558 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:02.556559 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:02.556564 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:02.558274 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:02.558634 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:02.558635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:02.558635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:02.558691 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:02.558922 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:02.558923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:02.558923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:02.559067 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:02.559144 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:02.559325 (+ 181us) server_negotiation.cc:299] Negotiation successful
0929 10:50:02.559405 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 10:50:02.559515 22269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:02.553182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:42475 (local address 127.20.80.194:59691)
0929 10:50:02.553335 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:02.553347 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:02.553412 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:02.553530 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:02.553531 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:02.553738 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:02.553807 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:02.553809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.554436 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.554437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:02.555308 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:02.555312 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.555690 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.555691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:02.555776 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:02.556136 (+ 360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:02.556140 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:02.556487 (+ 347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:02.558338 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:02.558339 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:02.558340 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:02.558514 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:02.558796 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:02.558797 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:02.558797 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:02.558856 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:02.559149 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:02.559150 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:02.559208 (+ 58us) client_negotiation.cc:769] Sending connection context
0929 10:50:02.559285 (+ 77us) client_negotiation.cc:240] Negotiation successful
0929 10:50:02.559417 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":88,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
I20260929 10:50:02.572453 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.195:0
--local_ip_for_outbound_sockets=127.20.80.195
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:42475
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:40303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:02.655170 22271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:02.655382 22271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:02.655406 22271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:02.656492 22271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:02.656526 22271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:02.656546 22271 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 10:50:02.656564 22271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.195
I20260929 10:50:02.658139 22271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:40303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.80.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.80.254:42475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.195
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:02.658330 22271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:02.658515 22271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 10:50:02.663875 22271 server_base.cc:1034] running on GCE node
W20260929 10:50:02.663746 22277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:02.663811 22279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:02.663746 22276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:02.664690 22271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:02.665010 22271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:02.666183 22271 hybrid_clock.cc:648] HybridClock initialized: now 1790679002666181 us; error 37 us; skew 500 ppm
Sep 29 10:50:02 dist-test-slave-vqch krb5kdc[21874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679002, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:02.668787 22271 init.cc:377] Logged in from keytab as kudu/127.20.80.195@KRBTEST.COM (short username kudu)
I20260929 10:50:02.669312 22271 webserver.cc:460] Webserver started at http://127.20.80.195:37225/ using document root <none> and password file <none>
I20260929 10:50:02.669548 22271 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:02.669617 22271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:02.669766 22271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:02.670852 22271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-2/data/instance:
uuid: "8632466c8c1d4c3d8398cfb4cd113a97"
format_stamp: "Formatted at 2026-09-29 10:50:02 on dist-test-slave-vqch"
server_key: "67f609446f54124b9c8502e899f45417"
server_key_iv: "10772822cf6625ea588b32076edc2294"
server_key_version: "encryptionkey@0"
I20260929 10:50:02.671221 22271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance:
uuid: "8632466c8c1d4c3d8398cfb4cd113a97"
format_stamp: "Formatted at 2026-09-29 10:50:02 on dist-test-slave-vqch"
server_key: "67f609446f54124b9c8502e899f45417"
server_key_iv: "10772822cf6625ea588b32076edc2294"
server_key_version: "encryptionkey@0"
I20260929 10:50:02.673064 22271 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260929 10:50:02.673899 22286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:02.674077 22271 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:02.674175 22271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-2/wal
uuid: "8632466c8c1d4c3d8398cfb4cd113a97"
format_stamp: "Formatted at 2026-09-29 10:50:02 on dist-test-slave-vqch"
server_key: "67f609446f54124b9c8502e899f45417"
server_key_iv: "10772822cf6625ea588b32076edc2294"
server_key_version: "encryptionkey@0"
I20260929 10:50:02.674259 22271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:02.690240 22271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:02.692396 22271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:02.692575 22271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:02.693246 22271 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:02.693317 22271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:02.693361 22271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:02.693389 22271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 10:50:02 dist-test-slave-vqch krb5kdc[21874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679002, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:02.700762 22271 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.195:40015
I20260929 10:50:02.700856 22401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.195:40015 every 8 connection(s)
I20260929 10:50:02.701176 22271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
I20260929 10:50:02.703043 21968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:02.693725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:60125 (local address 127.20.80.254:42475)
0929 10:50:02.693789 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:02.693792 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:02.694156 (+ 364us) server_negotiation.cc:407] Connection header received
0929 10:50:02.694901 (+ 745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:02.694902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:02.694913 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:02.695006 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:02.695945 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.696531 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.697749 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.697959 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.698044 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:02.700374 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:02.700375 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:02.700376 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:02.700382 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:02.701946 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:02.702296 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:02.702297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:02.702297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:02.702347 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:02.702538 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:02.702538 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:02.702538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:02.702630 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:02.702677 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:02.702920 (+ 243us) server_negotiation.cc:299] Negotiation successful
0929 10:50:02.702982 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260929 10:50:02.703194 22295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:02.693666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:42475 (local address 127.20.80.195:60125)
0929 10:50:02.693995 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:02.694040 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:02.694769 (+ 729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:02.695090 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:02.695094 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:02.695500 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:02.695823 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:02.695830 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.696629 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.696630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:02.697651 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:02.697654 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.698069 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.698070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:02.698229 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:02.698767 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:02.698774 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:02.700277 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:02.702016 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:02.702021 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:02.702022 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:02.702228 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:02.702405 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:02.702406 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:02.702406 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:02.702498 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:02.702680 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:02.702683 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:02.702808 (+ 125us) client_negotiation.cc:769] Sending connection context
0929 10:50:02.702889 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 10:50:02.703005 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":79,"threads_started":1}
I20260929 10:50:02.703647 22402 heartbeater.cc:347] Connected to a master server at 127.20.80.254:42475
I20260929 10:50:02.703740 22402 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:02.703914 22402 heartbeater.cc:511] Master 127.20.80.254:42475 requested a full tablet report, sending...
I20260929 10:50:02.704300 21919 ts_manager.cc:194] Registered new tserver with Master: 8632466c8c1d4c3d8398cfb4cd113a97 (127.20.80.195:40015)
I20260929 10:50:02.704800 21919 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.195@KRBTEST.COM'} at 127.20.80.195:60125
I20260929 10:50:02.707664 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 22271
I20260929 10:50:02.707748 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance
I20260929 10:50:02.707940 20803 external_mini_cluster.cc:1428] Setting key 4ddc236e457e3861b6af28c2b3de7e3d
I20260929 10:50:02.708844 20803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:50:02.710367 21968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:02.704749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:37439 (local address 127.20.80.254:42475)
0929 10:50:02.704788 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:02.704790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:02.704860 (+ 70us) server_negotiation.cc:407] Connection header received
0929 10:50:02.705022 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:02.705023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:02.705033 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:02.705080 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:02.705390 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.705914 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.706577 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.706770 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.706831 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:02.707578 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:02.707580 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:02.707580 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:02.707595 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:02.709260 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:02.709600 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:02.709601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:02.709602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:02.709663 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:02.709861 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:02.709862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:02.709863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:02.709973 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:02.710042 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:02.710204 (+ 162us) server_negotiation.cc:299] Negotiation successful
0929 10:50:02.710277 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 10:50:02.710388 22408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:02.704662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:42475 (local address 127.20.80.195:37439)
0929 10:50:02.704801 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:02.704812 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:02.704874 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:02.705099 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:02.705100 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:02.705256 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:02.705319 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:02.705320 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.705998 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.705999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:02.706488 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:02.706489 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.706840 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.706841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:02.706893 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:02.707235 (+ 342us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:02.707239 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:02.707490 (+ 251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:02.709344 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:02.709345 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:02.709346 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:02.709518 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:02.709729 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:02.709730 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:02.709730 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:02.709789 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:02.710047 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:02.710048 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:02.710093 (+ 45us) client_negotiation.cc:769] Sending connection context
0929 10:50:02.710178 (+ 85us) client_negotiation.cc:240] Negotiation successful
0929 10:50:02.710288 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":83,"threads_started":1}
Sep 29 10:50:02 dist-test-slave-vqch krb5kdc[21874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679002, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 10:50:02 dist-test-slave-vqch krb5kdc[21874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679002, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:02.731554 21968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:02.724603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37832 (local address 127.20.80.254:42475)
0929 10:50:02.724674 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:02.724676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:02.724684 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:50:02.724749 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:02.724750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:02.724759 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:02.724814 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:02.725423 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.725913 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.726617 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.726808 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.726860 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:02.729046 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:02.729047 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:02.729047 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:02.729053 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:02.730399 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:02.730832 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:02.730833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:02.730834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:02.730896 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:02.731098 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:02.731098 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:02.731098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:02.731202 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:02.731282 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:02.731395 (+ 113us) server_negotiation.cc:299] Negotiation successful
0929 10:50:02.731453 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 10:50:02.733268 21919 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:50:02.734843 21919 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:50:02.741869 22422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:02.737368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47516 (local address 127.20.80.195:40015)
0929 10:50:02.738069 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:02.738073 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:02.738084 (+ 11us) server_negotiation.cc:407] Connection header received
0929 10:50:02.738121 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:02.738123 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:02.738139 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:02.738211 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:02.738375 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.739187 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.740607 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.741342 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.741415 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:02.741482 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:02.741697 (+ 215us) server_negotiation.cc:299] Negotiation successful
0929 10:50:02.741742 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":86,"threads_started":1}
I20260929 10:50:02.741871 22419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:02.737224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:34123 (local address 127.0.0.1:52302)
0929 10:50:02.737547 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:02.737592 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:02.737661 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:02.737946 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:02.737949 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:02.737951 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:02.738019 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:02.738021 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.739353 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.739354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:02.740378 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:02.740380 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.741496 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.741498 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:02.741576 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:02.741591 (+ 15us) client_negotiation.cc:769] Sending connection context
0929 10:50:02.741686 (+ 95us) client_negotiation.cc:240] Negotiation successful
0929 10:50:02.741768 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":29,"thread_start_us":57,"threads_started":1}
I20260929 10:50:02.741880 22423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:02.737466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:34501 (local address 127.0.0.1:43034)
0929 10:50:02.738293 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:02.738302 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:02.738351 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:02.738551 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:02.738552 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:02.738554 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:02.738599 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:02.738601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.739775 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.739776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:02.740794 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:02.740796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.741611 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.741612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:02.741668 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:02.741672 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:50:02.741719 (+ 47us) client_negotiation.cc:240] Negotiation successful
0929 10:50:02.741768 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":807,"mutex_wait_us":3,"thread_start_us":63,"threads_started":1}
I20260929 10:50:02.742015 22420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:02.737257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:40015 (local address 127.0.0.1:47516)
0929 10:50:02.737552 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:02.737593 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:02.737678 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:02.738231 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:02.738232 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:02.738234 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:02.738296 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:02.738297 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.739275 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.739276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:02.740520 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:02.740523 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.741520 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.741521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:02.741590 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:02.741600 (+ 10us) client_negotiation.cc:769] Sending connection context
0929 10:50:02.741678 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 10:50:02.741751 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":91,"threads_started":1}
I20260929 10:50:02.742189 22421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:02.737347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52302 (local address 127.20.80.193:34123)
0929 10:50:02.737783 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:02.737787 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:02.737797 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:50:02.737839 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:02.737841 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:02.737853 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:02.737939 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:02.738105 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.739245 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.740466 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.741408 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.741473 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:02.741520 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:02.741993 (+ 473us) server_negotiation.cc:299] Negotiation successful
0929 10:50:02.742038 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":100,"threads_started":1}
I20260929 10:50:02.742795 22336 tablet_service.cc:1467] Processing CreateTablet for tablet a5c3c49a12ef4e1191e3897b4aa7648f (DEFAULT_TABLE table=test-table [id=525ac2e094d14995b57b185a2d12d75c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:02.743031 22056 tablet_service.cc:1467] Processing CreateTablet for tablet a5c3c49a12ef4e1191e3897b4aa7648f (DEFAULT_TABLE table=test-table [id=525ac2e094d14995b57b185a2d12d75c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:02.743160 22336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5c3c49a12ef4e1191e3897b4aa7648f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:02.743361 22056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5c3c49a12ef4e1191e3897b4aa7648f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:02.745803 22425 tablet_bootstrap.cc:492] T a5c3c49a12ef4e1191e3897b4aa7648f P 8632466c8c1d4c3d8398cfb4cd113a97: Bootstrap starting.
I20260929 10:50:02.746317 22426 tablet_bootstrap.cc:492] T a5c3c49a12ef4e1191e3897b4aa7648f P a1d4c6dbe57b4d4a867fd490b4926816: Bootstrap starting.
I20260929 10:50:02.746354 22424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:02.737569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43034 (local address 127.20.80.194:34501)
0929 10:50:02.737780 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:02.737785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:02.738407 (+ 622us) server_negotiation.cc:407] Connection header received
0929 10:50:02.738449 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:02.738451 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:02.738465 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:02.738536 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:02.738660 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.739679 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.740884 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.741431 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.741520 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:02.741579 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:02.741813 (+ 234us) server_negotiation.cc:299] Negotiation successful
0929 10:50:02.741857 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":87,"threads_started":1}
I20260929 10:50:02.746855 22425 tablet_bootstrap.cc:654] T a5c3c49a12ef4e1191e3897b4aa7648f P 8632466c8c1d4c3d8398cfb4cd113a97: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:02.746901 22426 tablet_bootstrap.cc:654] T a5c3c49a12ef4e1191e3897b4aa7648f P a1d4c6dbe57b4d4a867fd490b4926816: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:02.746870 22196 tablet_service.cc:1467] Processing CreateTablet for tablet a5c3c49a12ef4e1191e3897b4aa7648f (DEFAULT_TABLE table=test-table [id=525ac2e094d14995b57b185a2d12d75c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:02.747224 22196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5c3c49a12ef4e1191e3897b4aa7648f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:02.747222 22426 log.cc:824] T a5c3c49a12ef4e1191e3897b4aa7648f P a1d4c6dbe57b4d4a867fd490b4926816: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:02.747222 22425 log.cc:824] T a5c3c49a12ef4e1191e3897b4aa7648f P 8632466c8c1d4c3d8398cfb4cd113a97: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:02.747973 22425 tablet_bootstrap.cc:492] T a5c3c49a12ef4e1191e3897b4aa7648f P 8632466c8c1d4c3d8398cfb4cd113a97: No bootstrap required, opened a new log
I20260929 10:50:02.748044 22425 ts_tablet_manager.cc:1397] T a5c3c49a12ef4e1191e3897b4aa7648f P 8632466c8c1d4c3d8398cfb4cd113a97: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:50:02.748230 22426 tablet_bootstrap.cc:492] T a5c3c49a12ef4e1191e3897b4aa7648f P a1d4c6dbe57b4d4a867fd490b4926816: No bootstrap required, opened a new log
I20260929 10:50:02.748296 22426 ts_tablet_manager.cc:1397] T a5c3c49a12ef4e1191e3897b4aa7648f P a1d4c6dbe57b4d4a867fd490b4926816: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:50:02.750219 22426 raft_consensus.cc:348] T a5c3c49a12ef4e1191e3897b4aa7648f P a1d4c6dbe57b4d4a867fd490b4926816 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } }
I20260929 10:50:02.750219 22425 raft_consensus.cc:348] T a5c3c49a12ef4e1191e3897b4aa7648f P 8632466c8c1d4c3d8398cfb4cd113a97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } }
I20260929 10:50:02.750389 22426 raft_consensus.cc:374] T a5c3c49a12ef4e1191e3897b4aa7648f P a1d4c6dbe57b4d4a867fd490b4926816 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:02.750430 22425 raft_consensus.cc:374] T a5c3c49a12ef4e1191e3897b4aa7648f P 8632466c8c1d4c3d8398cfb4cd113a97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:02.750429 22426 raft_consensus.cc:729] T a5c3c49a12ef4e1191e3897b4aa7648f P a1d4c6dbe57b4d4a867fd490b4926816 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1d4c6dbe57b4d4a867fd490b4926816, State: Initialized, Role: FOLLOWER
I20260929 10:50:02.750463 22425 raft_consensus.cc:729] T a5c3c49a12ef4e1191e3897b4aa7648f P 8632466c8c1d4c3d8398cfb4cd113a97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8632466c8c1d4c3d8398cfb4cd113a97, State: Initialized, Role: FOLLOWER
I20260929 10:50:02.750564 22425 consensus_queue.cc:260] T a5c3c49a12ef4e1191e3897b4aa7648f P 8632466c8c1d4c3d8398cfb4cd113a97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } }
I20260929 10:50:02.750597 22426 consensus_queue.cc:260] T a5c3c49a12ef4e1191e3897b4aa7648f P a1d4c6dbe57b4d4a867fd490b4926816 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } }
I20260929 10:50:02.750844 22425 ts_tablet_manager.cc:1428] T a5c3c49a12ef4e1191e3897b4aa7648f P 8632466c8c1d4c3d8398cfb4cd113a97: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 10:50:02.750844 22426 ts_tablet_manager.cc:1428] T a5c3c49a12ef4e1191e3897b4aa7648f P a1d4c6dbe57b4d4a867fd490b4926816: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 10:50:02.750869 22122 heartbeater.cc:503] Master 127.20.80.254:42475 was elected leader, sending a full tablet report...
I20260929 10:50:02.750973 22402 heartbeater.cc:503] Master 127.20.80.254:42475 was elected leader, sending a full tablet report...
I20260929 10:50:02.751039 22429 tablet_bootstrap.cc:492] T a5c3c49a12ef4e1191e3897b4aa7648f P 634f61a9482d490c9a3e58ac5690f289: Bootstrap starting.
I20260929 10:50:02.751570 22429 tablet_bootstrap.cc:654] T a5c3c49a12ef4e1191e3897b4aa7648f P 634f61a9482d490c9a3e58ac5690f289: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:02.751946 22429 log.cc:824] T a5c3c49a12ef4e1191e3897b4aa7648f P 634f61a9482d490c9a3e58ac5690f289: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:02.752672 22429 tablet_bootstrap.cc:492] T a5c3c49a12ef4e1191e3897b4aa7648f P 634f61a9482d490c9a3e58ac5690f289: No bootstrap required, opened a new log
I20260929 10:50:02.752760 22429 ts_tablet_manager.cc:1397] T a5c3c49a12ef4e1191e3897b4aa7648f P 634f61a9482d490c9a3e58ac5690f289: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:50:02.754194 22429 raft_consensus.cc:348] T a5c3c49a12ef4e1191e3897b4aa7648f P 634f61a9482d490c9a3e58ac5690f289 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } }
I20260929 10:50:02.754317 22429 raft_consensus.cc:374] T a5c3c49a12ef4e1191e3897b4aa7648f P 634f61a9482d490c9a3e58ac5690f289 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:02.754339 22429 raft_consensus.cc:729] T a5c3c49a12ef4e1191e3897b4aa7648f P 634f61a9482d490c9a3e58ac5690f289 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 634f61a9482d490c9a3e58ac5690f289, State: Initialized, Role: FOLLOWER
I20260929 10:50:02.754415 22429 consensus_queue.cc:260] T a5c3c49a12ef4e1191e3897b4aa7648f P 634f61a9482d490c9a3e58ac5690f289 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } }
I20260929 10:50:02.754598 22429 ts_tablet_manager.cc:1428] T a5c3c49a12ef4e1191e3897b4aa7648f P 634f61a9482d490c9a3e58ac5690f289: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:50:02.754673 22262 heartbeater.cc:503] Master 127.20.80.254:42475 was elected leader, sending a full tablet report...
W20260929 10:50:02.795506 22263 tablet.cc:2369] T a5c3c49a12ef4e1191e3897b4aa7648f P 634f61a9482d490c9a3e58ac5690f289: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:50:02.805230 22430 raft_consensus.cc:482] T a5c3c49a12ef4e1191e3897b4aa7648f P 8632466c8c1d4c3d8398cfb4cd113a97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:50:02.805398 22430 raft_consensus.cc:504] T a5c3c49a12ef4e1191e3897b4aa7648f P 8632466c8c1d4c3d8398cfb4cd113a97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } }
I20260929 10:50:02.805841 22430 leader_election.cc:288] T a5c3c49a12ef4e1191e3897b4aa7648f P 8632466c8c1d4c3d8398cfb4cd113a97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1d4c6dbe57b4d4a867fd490b4926816 (127.20.80.193:34123), 634f61a9482d490c9a3e58ac5690f289 (127.20.80.194:34501)
I20260929 10:50:02.808902 22295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:02.806039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:34501 (local address 127.20.80.195:42839)
0929 10:50:02.806095 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:02.806107 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:02.806154 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:02.806418 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:02.806419 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:02.806421 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:02.806499 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:02.806501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.807272 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.807273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:02.808041 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:02.808044 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.808617 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.808618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:02.808706 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:02.808710 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:50:02.808784 (+ 74us) client_negotiation.cc:240] Negotiation successful
0929 10:50:02.808826 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260929 10:50:02.808900 22424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:02.806186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:42839 (local address 127.20.80.194:34501)
0929 10:50:02.806235 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:02.806237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:02.806247 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:50:02.806276 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:02.806276 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:02.806287 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:02.806358 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:02.806614 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.807178 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.808139 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.808529 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.808618 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:02.808628 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:02.808790 (+ 162us) server_negotiation.cc:299] Negotiation successful
0929 10:50:02.808830 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 10:50:02.809020 22434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:02.806038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:34123 (local address 127.20.80.195:50711)
0929 10:50:02.806408 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:02.806418 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:02.806479 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:02.806610 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:02.806611 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:02.806613 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:02.806674 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:02.806675 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.807396 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.807397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:02.808137 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:02.808139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.808729 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.808731 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:02.808815 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:02.808818 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:50:02.808883 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 10:50:02.808923 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":167,"thread_start_us":145,"threads_started":1}
I20260929 10:50:02.809013 22421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:02.806173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:50711 (local address 127.20.80.193:34123)
0929 10:50:02.806222 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:02.806225 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:02.806493 (+ 268us) server_negotiation.cc:407] Connection header received
0929 10:50:02.806526 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:02.806527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:02.806537 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:02.806589 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:02.806758 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.807272 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.808232 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.808638 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.808717 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:02.808725 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:02.808889 (+ 164us) server_negotiation.cc:299] Negotiation successful
0929 10:50:02.808928 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260929 10:50:02.809358 22216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c3c49a12ef4e1191e3897b4aa7648f" candidate_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "634f61a9482d490c9a3e58ac5690f289" is_pre_election: true
I20260929 10:50:02.809427 22076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c3c49a12ef4e1191e3897b4aa7648f" candidate_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" is_pre_election: true
I20260929 10:50:02.809533 22216 raft_consensus.cc:2454] T a5c3c49a12ef4e1191e3897b4aa7648f P 634f61a9482d490c9a3e58ac5690f289 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8632466c8c1d4c3d8398cfb4cd113a97 in term 0.
I20260929 10:50:02.809592 22076 raft_consensus.cc:2454] T a5c3c49a12ef4e1191e3897b4aa7648f P a1d4c6dbe57b4d4a867fd490b4926816 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8632466c8c1d4c3d8398cfb4cd113a97 in term 0.
I20260929 10:50:02.809757 22288 leader_election.cc:302] T a5c3c49a12ef4e1191e3897b4aa7648f P 8632466c8c1d4c3d8398cfb4cd113a97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 634f61a9482d490c9a3e58ac5690f289, 8632466c8c1d4c3d8398cfb4cd113a97; no voters:
I20260929 10:50:02.809932 22430 raft_consensus.cc:2781] T a5c3c49a12ef4e1191e3897b4aa7648f P 8632466c8c1d4c3d8398cfb4cd113a97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:50:02.810009 22430 raft_consensus.cc:482] T a5c3c49a12ef4e1191e3897b4aa7648f P 8632466c8c1d4c3d8398cfb4cd113a97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:50:02.810039 22430 raft_consensus.cc:3037] T a5c3c49a12ef4e1191e3897b4aa7648f P 8632466c8c1d4c3d8398cfb4cd113a97 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:02.823336 22430 raft_consensus.cc:504] T a5c3c49a12ef4e1191e3897b4aa7648f P 8632466c8c1d4c3d8398cfb4cd113a97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } }
I20260929 10:50:02.823757 22430 leader_election.cc:288] T a5c3c49a12ef4e1191e3897b4aa7648f P 8632466c8c1d4c3d8398cfb4cd113a97 [CANDIDATE]: Term 1 election: Requested vote from peers a1d4c6dbe57b4d4a867fd490b4926816 (127.20.80.193:34123), 634f61a9482d490c9a3e58ac5690f289 (127.20.80.194:34501)
I20260929 10:50:02.823822 22076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c3c49a12ef4e1191e3897b4aa7648f" candidate_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1d4c6dbe57b4d4a867fd490b4926816"
I20260929 10:50:02.823927 22076 raft_consensus.cc:3037] T a5c3c49a12ef4e1191e3897b4aa7648f P a1d4c6dbe57b4d4a867fd490b4926816 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:02.823895 22216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c3c49a12ef4e1191e3897b4aa7648f" candidate_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "634f61a9482d490c9a3e58ac5690f289"
I20260929 10:50:02.824008 22216 raft_consensus.cc:3037] T a5c3c49a12ef4e1191e3897b4aa7648f P 634f61a9482d490c9a3e58ac5690f289 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:02.824882 22076 raft_consensus.cc:2454] T a5c3c49a12ef4e1191e3897b4aa7648f P a1d4c6dbe57b4d4a867fd490b4926816 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8632466c8c1d4c3d8398cfb4cd113a97 in term 1.
I20260929 10:50:02.825148 22216 raft_consensus.cc:2454] T a5c3c49a12ef4e1191e3897b4aa7648f P 634f61a9482d490c9a3e58ac5690f289 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8632466c8c1d4c3d8398cfb4cd113a97 in term 1.
I20260929 10:50:02.825519 22289 leader_election.cc:302] T a5c3c49a12ef4e1191e3897b4aa7648f P 8632466c8c1d4c3d8398cfb4cd113a97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8632466c8c1d4c3d8398cfb4cd113a97, a1d4c6dbe57b4d4a867fd490b4926816; no voters:
I20260929 10:50:02.825628 22430 raft_consensus.cc:2781] T a5c3c49a12ef4e1191e3897b4aa7648f P 8632466c8c1d4c3d8398cfb4cd113a97 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:02.825778 22430 raft_consensus.cc:686] T a5c3c49a12ef4e1191e3897b4aa7648f P 8632466c8c1d4c3d8398cfb4cd113a97 [term 1 LEADER]: Becoming Leader. State: Replica: 8632466c8c1d4c3d8398cfb4cd113a97, State: Running, Role: LEADER
I20260929 10:50:02.825896 22430 consensus_queue.cc:237] T a5c3c49a12ef4e1191e3897b4aa7648f P 8632466c8c1d4c3d8398cfb4cd113a97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } }
I20260929 10:50:02.826938 21919 catalog_manager.cc:5495] T a5c3c49a12ef4e1191e3897b4aa7648f P 8632466c8c1d4c3d8398cfb4cd113a97 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8632466c8c1d4c3d8398cfb4cd113a97 (127.20.80.195). New cstate: current_term: 1 leader_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } health_report { overall_health: UNKNOWN } } }
I20260929 10:50:02.834277 22422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:02.831864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47518 (local address 127.20.80.195:40015)
0929 10:50:02.831927 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:02.831929 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:02.831937 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:50:02.831983 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:02.831984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:02.831999 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:02.832054 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:50:02.832242 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.832761 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.833553 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:02.833731 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:02.833785 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:02.833970 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:50:02.834056 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:50:02.834111 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:02.834181 (+ 70us) server_negotiation.cc:299] Negotiation successful
0929 10:50:02.834198 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 10:50:02.836184 22076 raft_consensus.cc:1261] T a5c3c49a12ef4e1191e3897b4aa7648f P a1d4c6dbe57b4d4a867fd490b4926816 [term 1 FOLLOWER]: Refusing update from remote peer 8632466c8c1d4c3d8398cfb4cd113a97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:02.836184 22216 raft_consensus.cc:1261] T a5c3c49a12ef4e1191e3897b4aa7648f P 634f61a9482d490c9a3e58ac5690f289 [term 1 FOLLOWER]: Refusing update from remote peer 8632466c8c1d4c3d8398cfb4cd113a97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:02.836513 22430 consensus_queue.cc:1035] T a5c3c49a12ef4e1191e3897b4aa7648f P 8632466c8c1d4c3d8398cfb4cd113a97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:50:02.836665 22435 consensus_queue.cc:1035] T a5c3c49a12ef4e1191e3897b4aa7648f P 8632466c8c1d4c3d8398cfb4cd113a97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:50:02.839800 22440 mvcc.cc:205] Tried to move back new op lower bound from 7334621195613917184 to 7334621195575746560. Current Snapshot: MvccSnapshot[applied={T|T < 7334621195613917184}]
I20260929 10:50:02.842605 21918 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:50:02.842729 21918 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:50:02.843566 21918 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2d9860e82bad4dda902f038e378b7410: Sending DeleteTablet for 3 replicas of tablet a5c3c49a12ef4e1191e3897b4aa7648f
I20260929 10:50:02.843962 22056 tablet_service.cc:1514] Processing DeleteTablet for tablet a5c3c49a12ef4e1191e3897b4aa7648f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:02 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:52302
I20260929 10:50:02.843951 22196 tablet_service.cc:1514] Processing DeleteTablet for tablet a5c3c49a12ef4e1191e3897b4aa7648f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:02 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:43034
I20260929 10:50:02.844177 22449 tablet_replica.cc:330] T a5c3c49a12ef4e1191e3897b4aa7648f P a1d4c6dbe57b4d4a867fd490b4926816: stopping tablet replica
I20260929 10:50:02.844215 22336 tablet_service.cc:1514] Processing DeleteTablet for tablet a5c3c49a12ef4e1191e3897b4aa7648f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:02 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:47516
I20260929 10:50:02.844295 22449 raft_consensus.cc:2229] T a5c3c49a12ef4e1191e3897b4aa7648f P a1d4c6dbe57b4d4a867fd490b4926816 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:50:02.844419 22449 raft_consensus.cc:2258] T a5c3c49a12ef4e1191e3897b4aa7648f P a1d4c6dbe57b4d4a867fd490b4926816 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:02.844666 22450 tablet_replica.cc:330] T a5c3c49a12ef4e1191e3897b4aa7648f P 8632466c8c1d4c3d8398cfb4cd113a97: stopping tablet replica
I20260929 10:50:02.844774 22450 raft_consensus.cc:2229] T a5c3c49a12ef4e1191e3897b4aa7648f P 8632466c8c1d4c3d8398cfb4cd113a97 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:50:02.844764 21918 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 10:50:02.844884 22450 raft_consensus.cc:2258] T a5c3c49a12ef4e1191e3897b4aa7648f P 8632466c8c1d4c3d8398cfb4cd113a97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:02.844879 22449 ts_tablet_manager.cc:1905] T a5c3c49a12ef4e1191e3897b4aa7648f P a1d4c6dbe57b4d4a867fd490b4926816: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260929 10:50:02.844897 21918 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:50:02.845826 22450 ts_tablet_manager.cc:1905] T a5c3c49a12ef4e1191e3897b4aa7648f P 8632466c8c1d4c3d8398cfb4cd113a97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:02.846760 22196 tablet_service.cc:1467] Processing CreateTablet for tablet a29139164a334d5a987efbb1e1c50054 (DEFAULT_TABLE table=test-table [id=39fed141b3df4cae9aa71bb16ded6922]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:02.846911 22196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a29139164a334d5a987efbb1e1c50054. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:02.847012 22336 tablet_service.cc:1467] Processing CreateTablet for tablet a29139164a334d5a987efbb1e1c50054 (DEFAULT_TABLE table=test-table [id=39fed141b3df4cae9aa71bb16ded6922]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:02.847128 22336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a29139164a334d5a987efbb1e1c50054. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:02.848006 22056 tablet_service.cc:1467] Processing CreateTablet for tablet a29139164a334d5a987efbb1e1c50054 (DEFAULT_TABLE table=test-table [id=39fed141b3df4cae9aa71bb16ded6922]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:02.848341 22429 tablet_bootstrap.cc:492] T a29139164a334d5a987efbb1e1c50054 P 634f61a9482d490c9a3e58ac5690f289: Bootstrap starting.
I20260929 10:50:02.846930 22448 tablet_replica.cc:330] T a5c3c49a12ef4e1191e3897b4aa7648f P 634f61a9482d490c9a3e58ac5690f289: stopping tablet replica
I20260929 10:50:02.848613 22448 raft_consensus.cc:2229] T a5c3c49a12ef4e1191e3897b4aa7648f P 634f61a9482d490c9a3e58ac5690f289 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:50:02.848739 22448 raft_consensus.cc:2258] T a5c3c49a12ef4e1191e3897b4aa7648f P 634f61a9482d490c9a3e58ac5690f289 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:02.849229 22448 ts_tablet_manager.cc:1905] T a5c3c49a12ef4e1191e3897b4aa7648f P 634f61a9482d490c9a3e58ac5690f289: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:02.849805 22429 tablet_bootstrap.cc:654] T a29139164a334d5a987efbb1e1c50054 P 634f61a9482d490c9a3e58ac5690f289: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:02.850430 22429 tablet_bootstrap.cc:492] T a29139164a334d5a987efbb1e1c50054 P 634f61a9482d490c9a3e58ac5690f289: No bootstrap required, opened a new log
I20260929 10:50:02.850494 22429 ts_tablet_manager.cc:1397] T a29139164a334d5a987efbb1e1c50054 P 634f61a9482d490c9a3e58ac5690f289: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260929 10:50:02.850656 22429 raft_consensus.cc:348] T a29139164a334d5a987efbb1e1c50054 P 634f61a9482d490c9a3e58ac5690f289 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } }
I20260929 10:50:02.850749 22429 raft_consensus.cc:374] T a29139164a334d5a987efbb1e1c50054 P 634f61a9482d490c9a3e58ac5690f289 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:02.850773 22429 raft_consensus.cc:729] T a29139164a334d5a987efbb1e1c50054 P 634f61a9482d490c9a3e58ac5690f289 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 634f61a9482d490c9a3e58ac5690f289, State: Initialized, Role: FOLLOWER
I20260929 10:50:02.850823 22429 consensus_queue.cc:260] T a29139164a334d5a987efbb1e1c50054 P 634f61a9482d490c9a3e58ac5690f289 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } }
I20260929 10:50:02.850905 22429 ts_tablet_manager.cc:1428] T a29139164a334d5a987efbb1e1c50054 P 634f61a9482d490c9a3e58ac5690f289: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260929 10:50:02.886850 22448 ts_tablet_manager.cc:1918] T a5c3c49a12ef4e1191e3897b4aa7648f P 634f61a9482d490c9a3e58ac5690f289: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:50:02.886986 22448 log.cc:1196] T a5c3c49a12ef4e1191e3897b4aa7648f P 634f61a9482d490c9a3e58ac5690f289: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-1/wal/wals/a5c3c49a12ef4e1191e3897b4aa7648f
I20260929 10:50:02.887256 22056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a29139164a334d5a987efbb1e1c50054. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:02.887256 22450 ts_tablet_manager.cc:1918] T a5c3c49a12ef4e1191e3897b4aa7648f P 8632466c8c1d4c3d8398cfb4cd113a97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:50:02.887291 22448 ts_tablet_manager.cc:1939] T a5c3c49a12ef4e1191e3897b4aa7648f P 634f61a9482d490c9a3e58ac5690f289: Deleting consensus metadata
I20260929 10:50:02.887347 22450 log.cc:1196] T a5c3c49a12ef4e1191e3897b4aa7648f P 8632466c8c1d4c3d8398cfb4cd113a97: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-2/wal/wals/a5c3c49a12ef4e1191e3897b4aa7648f
I20260929 10:50:02.887624 22450 ts_tablet_manager.cc:1939] T a5c3c49a12ef4e1191e3897b4aa7648f P 8632466c8c1d4c3d8398cfb4cd113a97: Deleting consensus metadata
I20260929 10:50:02.888226 22425 tablet_bootstrap.cc:492] T a29139164a334d5a987efbb1e1c50054 P 8632466c8c1d4c3d8398cfb4cd113a97: Bootstrap starting.
I20260929 10:50:02.888376 22449 ts_tablet_manager.cc:1918] T a5c3c49a12ef4e1191e3897b4aa7648f P a1d4c6dbe57b4d4a867fd490b4926816: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:50:02.888470 22449 log.cc:1196] T a5c3c49a12ef4e1191e3897b4aa7648f P a1d4c6dbe57b4d4a867fd490b4926816: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-0/wal/wals/a5c3c49a12ef4e1191e3897b4aa7648f
I20260929 10:50:02.888651 22425 tablet_bootstrap.cc:654] T a29139164a334d5a987efbb1e1c50054 P 8632466c8c1d4c3d8398cfb4cd113a97: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:02.888825 22449 ts_tablet_manager.cc:1939] T a5c3c49a12ef4e1191e3897b4aa7648f P a1d4c6dbe57b4d4a867fd490b4926816: Deleting consensus metadata
I20260929 10:50:02.889066 22426 tablet_bootstrap.cc:492] T a29139164a334d5a987efbb1e1c50054 P a1d4c6dbe57b4d4a867fd490b4926816: Bootstrap starting.
I20260929 10:50:02.889060 21904 catalog_manager.cc:4841] TS 8632466c8c1d4c3d8398cfb4cd113a97 (127.20.80.195:40015): tablet a5c3c49a12ef4e1191e3897b4aa7648f (table test-table [id=525ac2e094d14995b57b185a2d12d75c]) successfully deleted
I20260929 10:50:02.889353 21905 catalog_manager.cc:4841] TS a1d4c6dbe57b4d4a867fd490b4926816 (127.20.80.193:34123): tablet a5c3c49a12ef4e1191e3897b4aa7648f (table test-table [id=525ac2e094d14995b57b185a2d12d75c]) successfully deleted
I20260929 10:50:02.889503 22426 tablet_bootstrap.cc:654] T a29139164a334d5a987efbb1e1c50054 P a1d4c6dbe57b4d4a867fd490b4926816: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:02.889690 22425 tablet_bootstrap.cc:492] T a29139164a334d5a987efbb1e1c50054 P 8632466c8c1d4c3d8398cfb4cd113a97: No bootstrap required, opened a new log
I20260929 10:50:02.889731 22425 ts_tablet_manager.cc:1397] T a29139164a334d5a987efbb1e1c50054 P 8632466c8c1d4c3d8398cfb4cd113a97: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260929 10:50:02.889882 22425 raft_consensus.cc:348] T a29139164a334d5a987efbb1e1c50054 P 8632466c8c1d4c3d8398cfb4cd113a97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } }
I20260929 10:50:02.889964 22425 raft_consensus.cc:374] T a29139164a334d5a987efbb1e1c50054 P 8632466c8c1d4c3d8398cfb4cd113a97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:02.889981 22425 raft_consensus.cc:729] T a29139164a334d5a987efbb1e1c50054 P 8632466c8c1d4c3d8398cfb4cd113a97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8632466c8c1d4c3d8398cfb4cd113a97, State: Initialized, Role: FOLLOWER
I20260929 10:50:02.890023 22425 consensus_queue.cc:260] T a29139164a334d5a987efbb1e1c50054 P 8632466c8c1d4c3d8398cfb4cd113a97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } }
I20260929 10:50:02.890125 22425 ts_tablet_manager.cc:1428] T a29139164a334d5a987efbb1e1c50054 P 8632466c8c1d4c3d8398cfb4cd113a97: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:02.890123 22426 tablet_bootstrap.cc:492] T a29139164a334d5a987efbb1e1c50054 P a1d4c6dbe57b4d4a867fd490b4926816: No bootstrap required, opened a new log
I20260929 10:50:02.890182 22426 ts_tablet_manager.cc:1397] T a29139164a334d5a987efbb1e1c50054 P a1d4c6dbe57b4d4a867fd490b4926816: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:02.890345 22426 raft_consensus.cc:348] T a29139164a334d5a987efbb1e1c50054 P a1d4c6dbe57b4d4a867fd490b4926816 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } }
I20260929 10:50:02.890385 21904 catalog_manager.cc:4841] TS 634f61a9482d490c9a3e58ac5690f289 (127.20.80.194:34501): tablet a5c3c49a12ef4e1191e3897b4aa7648f (table test-table [id=525ac2e094d14995b57b185a2d12d75c]) successfully deleted
I20260929 10:50:02.890411 22426 raft_consensus.cc:374] T a29139164a334d5a987efbb1e1c50054 P a1d4c6dbe57b4d4a867fd490b4926816 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:02.890430 22426 raft_consensus.cc:729] T a29139164a334d5a987efbb1e1c50054 P a1d4c6dbe57b4d4a867fd490b4926816 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1d4c6dbe57b4d4a867fd490b4926816, State: Initialized, Role: FOLLOWER
I20260929 10:50:02.890476 22426 consensus_queue.cc:260] T a29139164a334d5a987efbb1e1c50054 P a1d4c6dbe57b4d4a867fd490b4926816 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } }
I20260929 10:50:02.890585 22426 ts_tablet_manager.cc:1428] T a29139164a334d5a987efbb1e1c50054 P a1d4c6dbe57b4d4a867fd490b4926816: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260929 10:50:02.952316 22403 tablet.cc:2369] T a29139164a334d5a987efbb1e1c50054 P 8632466c8c1d4c3d8398cfb4cd113a97: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:50:03.140082 22123 tablet.cc:2369] T a29139164a334d5a987efbb1e1c50054 P a1d4c6dbe57b4d4a867fd490b4926816: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:50:03.274571 22433 raft_consensus.cc:482] T a29139164a334d5a987efbb1e1c50054 P 634f61a9482d490c9a3e58ac5690f289 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:50:03.274688 22433 raft_consensus.cc:504] T a29139164a334d5a987efbb1e1c50054 P 634f61a9482d490c9a3e58ac5690f289 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } }
I20260929 10:50:03.275043 22433 leader_election.cc:288] T a29139164a334d5a987efbb1e1c50054 P 634f61a9482d490c9a3e58ac5690f289 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1d4c6dbe57b4d4a867fd490b4926816 (127.20.80.193:34123), 8632466c8c1d4c3d8398cfb4cd113a97 (127.20.80.195:40015)
I20260929 10:50:03.278358 22455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:03.275268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:34123 (local address 127.20.80.194:56493)
0929 10:50:03.275425 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:03.275437 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:03.275509 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:03.275652 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:03.275653 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:03.275657 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:03.275776 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:03.275780 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.276523 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.276524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:03.277524 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:03.277527 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.278071 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.278072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:03.278150 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:03.278154 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:50:03.278216 (+ 62us) client_negotiation.cc:240] Negotiation successful
0929 10:50:03.278259 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":38,"thread_start_us":63,"threads_started":1}
I20260929 10:50:03.278343 22421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:03.275345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:56493 (local address 127.20.80.193:34123)
0929 10:50:03.275399 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:03.275402 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:03.275520 (+ 118us) server_negotiation.cc:407] Connection header received
0929 10:50:03.275560 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:03.275561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:03.275574 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:03.275634 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:03.275859 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.276436 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.277615 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.277987 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.278072 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:03.278081 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:03.278237 (+ 156us) server_negotiation.cc:299] Negotiation successful
0929 10:50:03.278275 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 10:50:03.278470 22422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:03.275383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:46321 (local address 127.20.80.195:40015)
0929 10:50:03.275437 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:03.275439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:03.275661 (+ 222us) server_negotiation.cc:407] Connection header received
0929 10:50:03.275692 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:03.275694 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:03.275708 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:03.275763 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:03.275943 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.276412 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.277700 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.278069 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.278154 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:03.278162 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:03.278359 (+ 197us) server_negotiation.cc:299] Negotiation successful
0929 10:50:03.278400 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260929 10:50:03.278478 22454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:03.275271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:40015 (local address 127.20.80.194:46321)
0929 10:50:03.275538 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:03.275549 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:03.275617 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:03.275786 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:03.275787 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:03.275790 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:03.275871 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:03.275873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.276539 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.276540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:03.277531 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:03.277534 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.278164 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.278165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:03.278246 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:03.278249 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:50:03.278340 (+ 91us) client_negotiation.cc:240] Negotiation successful
0929 10:50:03.278386 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":47,"thread_start_us":75,"threads_started":1}
I20260929 10:50:03.278806 22076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a29139164a334d5a987efbb1e1c50054" candidate_uuid: "634f61a9482d490c9a3e58ac5690f289" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" is_pre_election: true
I20260929 10:50:03.278923 22076 raft_consensus.cc:2454] T a29139164a334d5a987efbb1e1c50054 P a1d4c6dbe57b4d4a867fd490b4926816 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 634f61a9482d490c9a3e58ac5690f289 in term 0.
I20260929 10:50:03.279024 22356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a29139164a334d5a987efbb1e1c50054" candidate_uuid: "634f61a9482d490c9a3e58ac5690f289" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" is_pre_election: true
I20260929 10:50:03.279109 22149 leader_election.cc:302] T a29139164a334d5a987efbb1e1c50054 P 634f61a9482d490c9a3e58ac5690f289 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 634f61a9482d490c9a3e58ac5690f289, a1d4c6dbe57b4d4a867fd490b4926816; no voters:
I20260929 10:50:03.279162 22356 raft_consensus.cc:2454] T a29139164a334d5a987efbb1e1c50054 P 8632466c8c1d4c3d8398cfb4cd113a97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 634f61a9482d490c9a3e58ac5690f289 in term 0.
I20260929 10:50:03.279326 22433 raft_consensus.cc:2781] T a29139164a334d5a987efbb1e1c50054 P 634f61a9482d490c9a3e58ac5690f289 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:50:03.279364 22433 raft_consensus.cc:482] T a29139164a334d5a987efbb1e1c50054 P 634f61a9482d490c9a3e58ac5690f289 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:50:03.279378 22433 raft_consensus.cc:3037] T a29139164a334d5a987efbb1e1c50054 P 634f61a9482d490c9a3e58ac5690f289 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:03.280022 22433 raft_consensus.cc:504] T a29139164a334d5a987efbb1e1c50054 P 634f61a9482d490c9a3e58ac5690f289 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } }
I20260929 10:50:03.280169 22433 leader_election.cc:288] T a29139164a334d5a987efbb1e1c50054 P 634f61a9482d490c9a3e58ac5690f289 [CANDIDATE]: Term 1 election: Requested vote from peers a1d4c6dbe57b4d4a867fd490b4926816 (127.20.80.193:34123), 8632466c8c1d4c3d8398cfb4cd113a97 (127.20.80.195:40015)
I20260929 10:50:03.280341 22076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a29139164a334d5a987efbb1e1c50054" candidate_uuid: "634f61a9482d490c9a3e58ac5690f289" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1d4c6dbe57b4d4a867fd490b4926816"
I20260929 10:50:03.280364 22356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a29139164a334d5a987efbb1e1c50054" candidate_uuid: "634f61a9482d490c9a3e58ac5690f289" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8632466c8c1d4c3d8398cfb4cd113a97"
I20260929 10:50:03.280435 22356 raft_consensus.cc:3037] T a29139164a334d5a987efbb1e1c50054 P 8632466c8c1d4c3d8398cfb4cd113a97 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:03.280438 22076 raft_consensus.cc:3037] T a29139164a334d5a987efbb1e1c50054 P a1d4c6dbe57b4d4a867fd490b4926816 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:03.281065 22076 raft_consensus.cc:2454] T a29139164a334d5a987efbb1e1c50054 P a1d4c6dbe57b4d4a867fd490b4926816 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 634f61a9482d490c9a3e58ac5690f289 in term 1.
I20260929 10:50:03.281340 22356 raft_consensus.cc:2454] T a29139164a334d5a987efbb1e1c50054 P 8632466c8c1d4c3d8398cfb4cd113a97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 634f61a9482d490c9a3e58ac5690f289 in term 1.
I20260929 10:50:03.281535 22149 leader_election.cc:302] T a29139164a334d5a987efbb1e1c50054 P 634f61a9482d490c9a3e58ac5690f289 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 634f61a9482d490c9a3e58ac5690f289, a1d4c6dbe57b4d4a867fd490b4926816; no voters:
I20260929 10:50:03.281648 22433 raft_consensus.cc:2781] T a29139164a334d5a987efbb1e1c50054 P 634f61a9482d490c9a3e58ac5690f289 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:03.281826 22433 raft_consensus.cc:686] T a29139164a334d5a987efbb1e1c50054 P 634f61a9482d490c9a3e58ac5690f289 [term 1 LEADER]: Becoming Leader. State: Replica: 634f61a9482d490c9a3e58ac5690f289, State: Running, Role: LEADER
I20260929 10:50:03.281915 22433 consensus_queue.cc:237] T a29139164a334d5a987efbb1e1c50054 P 634f61a9482d490c9a3e58ac5690f289 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } }
I20260929 10:50:03.282615 21919 catalog_manager.cc:5495] T a29139164a334d5a987efbb1e1c50054 P 634f61a9482d490c9a3e58ac5690f289 reported cstate change: term changed from 0 to 1, leader changed from <none> to 634f61a9482d490c9a3e58ac5690f289 (127.20.80.194). New cstate: current_term: 1 leader_uuid: "634f61a9482d490c9a3e58ac5690f289" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } health_report { overall_health: UNKNOWN } } }
Sep 29 10:50:03 dist-test-slave-vqch krb5kdc[21874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679002, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:03.313350 22463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:03.305727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:42475 (local address 127.0.0.1:53530)
0929 10:50:03.305930 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:03.305949 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:03.306018 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:03.306205 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:03.306207 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:03.306601 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:03.306667 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:03.306670 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.307359 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.307360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:03.307997 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:03.307999 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.308497 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.308498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:03.308548 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:03.309172 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:03.309178 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:03.310594 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:03.312243 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:03.312251 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:03.312253 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:03.312462 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:03.312660 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:03.312661 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:03.312661 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:03.312758 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:03.313043 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:03.313052 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:03.313106 (+ 54us) client_negotiation.cc:769] Sending connection context
0929 10:50:03.313173 (+ 67us) client_negotiation.cc:240] Negotiation successful
0929 10:50:03.313274 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":96,"threads_started":1}
I20260929 10:50:03.313422 22464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:03.305793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53530 (local address 127.20.80.254:42475)
0929 10:50:03.305967 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:03.305969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:03.305998 (+ 29us) server_negotiation.cc:407] Connection header received
0929 10:50:03.306077 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:03.306078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:03.306087 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:03.306151 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:03.306747 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.307274 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.308090 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.308397 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.308471 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:03.310670 (+ 2199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:03.310672 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:03.310672 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:03.310679 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:03.312154 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:03.312526 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:03.312527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:03.312527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:03.312580 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:03.312830 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:03.312830 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:03.312831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:03.312930 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:03.313023 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:03.313229 (+ 206us) server_negotiation.cc:299] Negotiation successful
0929 10:50:03.313337 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":149,"thread_start_us":115,"threads_started":1}
I20260929 10:50:03.315626 21919 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:53530:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 10:50:03.315810 21919 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:50:03.317301 22056 tablet_service.cc:1467] Processing CreateTablet for tablet 64bd5b74f62941158bf836e4ae528807 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2db0a0a7ec5a4dc09d84c0f55460ed0e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 10:50:03.317472 22056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64bd5b74f62941158bf836e4ae528807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:03.317477 22196 tablet_service.cc:1467] Processing CreateTablet for tablet 64bd5b74f62941158bf836e4ae528807 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2db0a0a7ec5a4dc09d84c0f55460ed0e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 10:50:03.317471 22336 tablet_service.cc:1467] Processing CreateTablet for tablet 64bd5b74f62941158bf836e4ae528807 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2db0a0a7ec5a4dc09d84c0f55460ed0e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 10:50:03.317611 22196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64bd5b74f62941158bf836e4ae528807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:03.317636 22336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64bd5b74f62941158bf836e4ae528807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:03.318760 22425 tablet_bootstrap.cc:492] T 64bd5b74f62941158bf836e4ae528807 P 8632466c8c1d4c3d8398cfb4cd113a97: Bootstrap starting.
I20260929 10:50:03.319026 22426 tablet_bootstrap.cc:492] T 64bd5b74f62941158bf836e4ae528807 P a1d4c6dbe57b4d4a867fd490b4926816: Bootstrap starting.
I20260929 10:50:03.319200 22425 tablet_bootstrap.cc:654] T 64bd5b74f62941158bf836e4ae528807 P 8632466c8c1d4c3d8398cfb4cd113a97: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:03.319419 22426 tablet_bootstrap.cc:654] T 64bd5b74f62941158bf836e4ae528807 P a1d4c6dbe57b4d4a867fd490b4926816: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:03.319763 22429 tablet_bootstrap.cc:492] T 64bd5b74f62941158bf836e4ae528807 P 634f61a9482d490c9a3e58ac5690f289: Bootstrap starting.
I20260929 10:50:03.319834 22425 tablet_bootstrap.cc:492] T 64bd5b74f62941158bf836e4ae528807 P 8632466c8c1d4c3d8398cfb4cd113a97: No bootstrap required, opened a new log
I20260929 10:50:03.319878 22425 ts_tablet_manager.cc:1397] T 64bd5b74f62941158bf836e4ae528807 P 8632466c8c1d4c3d8398cfb4cd113a97: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 10:50:03.319998 22425 raft_consensus.cc:348] T 64bd5b74f62941158bf836e4ae528807 P 8632466c8c1d4c3d8398cfb4cd113a97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } }
I20260929 10:50:03.320046 22426 tablet_bootstrap.cc:492] T 64bd5b74f62941158bf836e4ae528807 P a1d4c6dbe57b4d4a867fd490b4926816: No bootstrap required, opened a new log
I20260929 10:50:03.320063 22425 raft_consensus.cc:374] T 64bd5b74f62941158bf836e4ae528807 P 8632466c8c1d4c3d8398cfb4cd113a97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:03.320081 22426 ts_tablet_manager.cc:1397] T 64bd5b74f62941158bf836e4ae528807 P a1d4c6dbe57b4d4a867fd490b4926816: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:03.320088 22425 raft_consensus.cc:729] T 64bd5b74f62941158bf836e4ae528807 P 8632466c8c1d4c3d8398cfb4cd113a97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8632466c8c1d4c3d8398cfb4cd113a97, State: Initialized, Role: FOLLOWER
I20260929 10:50:03.320149 22425 consensus_queue.cc:260] T 64bd5b74f62941158bf836e4ae528807 P 8632466c8c1d4c3d8398cfb4cd113a97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } }
I20260929 10:50:03.320180 22429 tablet_bootstrap.cc:654] T 64bd5b74f62941158bf836e4ae528807 P 634f61a9482d490c9a3e58ac5690f289: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:03.320264 22425 ts_tablet_manager.cc:1428] T 64bd5b74f62941158bf836e4ae528807 P 8632466c8c1d4c3d8398cfb4cd113a97: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:03.320250 22426 raft_consensus.cc:348] T 64bd5b74f62941158bf836e4ae528807 P a1d4c6dbe57b4d4a867fd490b4926816 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } }
I20260929 10:50:03.320320 22426 raft_consensus.cc:374] T 64bd5b74f62941158bf836e4ae528807 P a1d4c6dbe57b4d4a867fd490b4926816 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:03.320340 22426 raft_consensus.cc:729] T 64bd5b74f62941158bf836e4ae528807 P a1d4c6dbe57b4d4a867fd490b4926816 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1d4c6dbe57b4d4a867fd490b4926816, State: Initialized, Role: FOLLOWER
I20260929 10:50:03.320281 22443 tablet_replica.cc:439] T 64bd5b74f62941158bf836e4ae528807 P 8632466c8c1d4c3d8398cfb4cd113a97: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } } }
I20260929 10:50:03.320371 22443 tablet_replica.cc:442] T 64bd5b74f62941158bf836e4ae528807 P 8632466c8c1d4c3d8398cfb4cd113a97: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:50:03.320382 22426 consensus_queue.cc:260] T 64bd5b74f62941158bf836e4ae528807 P a1d4c6dbe57b4d4a867fd490b4926816 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } }
I20260929 10:50:03.320485 22426 ts_tablet_manager.cc:1428] T 64bd5b74f62941158bf836e4ae528807 P a1d4c6dbe57b4d4a867fd490b4926816: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:03.320772 22429 tablet_bootstrap.cc:492] T 64bd5b74f62941158bf836e4ae528807 P 634f61a9482d490c9a3e58ac5690f289: No bootstrap required, opened a new log
I20260929 10:50:03.320737 22431 tablet_replica.cc:439] T 64bd5b74f62941158bf836e4ae528807 P a1d4c6dbe57b4d4a867fd490b4926816: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } } }
I20260929 10:50:03.320828 22429 ts_tablet_manager.cc:1397] T 64bd5b74f62941158bf836e4ae528807 P 634f61a9482d490c9a3e58ac5690f289: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 10:50:03.320832 22431 tablet_replica.cc:442] T 64bd5b74f62941158bf836e4ae528807 P a1d4c6dbe57b4d4a867fd490b4926816: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:50:03.320959 22429 raft_consensus.cc:348] T 64bd5b74f62941158bf836e4ae528807 P 634f61a9482d490c9a3e58ac5690f289 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } }
I20260929 10:50:03.321069 22429 raft_consensus.cc:374] T 64bd5b74f62941158bf836e4ae528807 P 634f61a9482d490c9a3e58ac5690f289 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:03.321094 22429 raft_consensus.cc:729] T 64bd5b74f62941158bf836e4ae528807 P 634f61a9482d490c9a3e58ac5690f289 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 634f61a9482d490c9a3e58ac5690f289, State: Initialized, Role: FOLLOWER
I20260929 10:50:03.321138 22429 consensus_queue.cc:260] T 64bd5b74f62941158bf836e4ae528807 P 634f61a9482d490c9a3e58ac5690f289 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } }
I20260929 10:50:03.321221 22429 ts_tablet_manager.cc:1428] T 64bd5b74f62941158bf836e4ae528807 P 634f61a9482d490c9a3e58ac5690f289: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:03.321293 22433 tablet_replica.cc:439] T 64bd5b74f62941158bf836e4ae528807 P 634f61a9482d490c9a3e58ac5690f289: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } } }
I20260929 10:50:03.321362 22433 tablet_replica.cc:442] T 64bd5b74f62941158bf836e4ae528807 P 634f61a9482d490c9a3e58ac5690f289: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:50:03.463405 22431 raft_consensus.cc:482] T 64bd5b74f62941158bf836e4ae528807 P a1d4c6dbe57b4d4a867fd490b4926816 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:50:03.463529 22431 raft_consensus.cc:504] T 64bd5b74f62941158bf836e4ae528807 P a1d4c6dbe57b4d4a867fd490b4926816 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } }
I20260929 10:50:03.463989 22431 leader_election.cc:288] T 64bd5b74f62941158bf836e4ae528807 P a1d4c6dbe57b4d4a867fd490b4926816 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 634f61a9482d490c9a3e58ac5690f289 (127.20.80.194:34501), 8632466c8c1d4c3d8398cfb4cd113a97 (127.20.80.195:40015)
I20260929 10:50:03.467298 22422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:03.464533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:43461 (local address 127.20.80.195:40015)
0929 10:50:03.464607 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:03.464609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:03.464621 (+ 12us) server_negotiation.cc:407] Connection header received
0929 10:50:03.464674 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:03.464675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:03.464685 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:03.464743 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:03.464900 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.465448 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.466568 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.466944 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.467029 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:03.467039 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:03.467180 (+ 141us) server_negotiation.cc:299] Negotiation successful
0929 10:50:03.467219 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 10:50:03.467335 22469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:03.464272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:55739 (local address 127.20.80.194:34501)
0929 10:50:03.464402 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:03.464408 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:03.464417 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:50:03.464522 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:03.464523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:03.464534 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:03.464610 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:03.464769 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.465353 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.466582 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.466963 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.467049 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:03.467058 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:03.467213 (+ 155us) server_negotiation.cc:299] Negotiation successful
0929 10:50:03.467262 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":71,"threads_started":1}
I20260929 10:50:03.467357 22468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:03.464201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:34501 (local address 127.20.80.193:55739)
0929 10:50:03.464362 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:03.464375 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:03.464434 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:03.464619 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:03.464621 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:03.464624 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:03.464692 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:03.464694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.465569 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.465570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:03.466474 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:03.466476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.467049 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.467050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:03.467121 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:03.467124 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:50:03.467190 (+ 66us) client_negotiation.cc:240] Negotiation successful
0929 10:50:03.467235 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":86,"thread_start_us":111,"threads_started":1}
I20260929 10:50:03.467368 22470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:03.464443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:40015 (local address 127.20.80.193:43461)
0929 10:50:03.464558 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:03.464568 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:03.464611 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:03.464748 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:03.464749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:03.464752 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:03.464827 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:03.464828 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.465602 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.465603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:03.466469 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:03.466472 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.467038 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.467039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:03.467104 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:03.467107 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:50:03.467175 (+ 68us) client_negotiation.cc:240] Negotiation successful
0929 10:50:03.467219 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"mutex_wait_us":26,"thread_start_us":41,"threads_started":1}
I20260929 10:50:03.467628 22356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64bd5b74f62941158bf836e4ae528807" candidate_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" is_pre_election: true
I20260929 10:50:03.467656 22216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64bd5b74f62941158bf836e4ae528807" candidate_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "634f61a9482d490c9a3e58ac5690f289" is_pre_election: true
I20260929 10:50:03.467741 22216 raft_consensus.cc:2454] T 64bd5b74f62941158bf836e4ae528807 P 634f61a9482d490c9a3e58ac5690f289 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1d4c6dbe57b4d4a867fd490b4926816 in term 0.
I20260929 10:50:03.467741 22356 raft_consensus.cc:2454] T 64bd5b74f62941158bf836e4ae528807 P 8632466c8c1d4c3d8398cfb4cd113a97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1d4c6dbe57b4d4a867fd490b4926816 in term 0.
I20260929 10:50:03.467906 22008 leader_election.cc:302] T 64bd5b74f62941158bf836e4ae528807 P a1d4c6dbe57b4d4a867fd490b4926816 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 634f61a9482d490c9a3e58ac5690f289, a1d4c6dbe57b4d4a867fd490b4926816; no voters:
I20260929 10:50:03.468048 22431 raft_consensus.cc:2781] T 64bd5b74f62941158bf836e4ae528807 P a1d4c6dbe57b4d4a867fd490b4926816 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:50:03.468115 22431 raft_consensus.cc:482] T 64bd5b74f62941158bf836e4ae528807 P a1d4c6dbe57b4d4a867fd490b4926816 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:50:03.468144 22431 raft_consensus.cc:3037] T 64bd5b74f62941158bf836e4ae528807 P a1d4c6dbe57b4d4a867fd490b4926816 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:03.468879 22431 raft_consensus.cc:504] T 64bd5b74f62941158bf836e4ae528807 P a1d4c6dbe57b4d4a867fd490b4926816 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } }
I20260929 10:50:03.469081 22431 leader_election.cc:288] T 64bd5b74f62941158bf836e4ae528807 P a1d4c6dbe57b4d4a867fd490b4926816 [CANDIDATE]: Term 1 election: Requested vote from peers 634f61a9482d490c9a3e58ac5690f289 (127.20.80.194:34501), 8632466c8c1d4c3d8398cfb4cd113a97 (127.20.80.195:40015)
I20260929 10:50:03.469264 22216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64bd5b74f62941158bf836e4ae528807" candidate_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "634f61a9482d490c9a3e58ac5690f289"
I20260929 10:50:03.469287 22356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64bd5b74f62941158bf836e4ae528807" candidate_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8632466c8c1d4c3d8398cfb4cd113a97"
I20260929 10:50:03.469352 22356 raft_consensus.cc:3037] T 64bd5b74f62941158bf836e4ae528807 P 8632466c8c1d4c3d8398cfb4cd113a97 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:03.469352 22216 raft_consensus.cc:3037] T 64bd5b74f62941158bf836e4ae528807 P 634f61a9482d490c9a3e58ac5690f289 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:03.470079 22356 raft_consensus.cc:2454] T 64bd5b74f62941158bf836e4ae528807 P 8632466c8c1d4c3d8398cfb4cd113a97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1d4c6dbe57b4d4a867fd490b4926816 in term 1.
I20260929 10:50:03.470079 22216 raft_consensus.cc:2454] T 64bd5b74f62941158bf836e4ae528807 P 634f61a9482d490c9a3e58ac5690f289 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1d4c6dbe57b4d4a867fd490b4926816 in term 1.
I20260929 10:50:03.470307 22008 leader_election.cc:302] T 64bd5b74f62941158bf836e4ae528807 P a1d4c6dbe57b4d4a867fd490b4926816 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8632466c8c1d4c3d8398cfb4cd113a97, a1d4c6dbe57b4d4a867fd490b4926816; no voters:
I20260929 10:50:03.470410 22431 raft_consensus.cc:2781] T 64bd5b74f62941158bf836e4ae528807 P a1d4c6dbe57b4d4a867fd490b4926816 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:03.470556 22431 raft_consensus.cc:686] T 64bd5b74f62941158bf836e4ae528807 P a1d4c6dbe57b4d4a867fd490b4926816 [term 1 LEADER]: Becoming Leader. State: Replica: a1d4c6dbe57b4d4a867fd490b4926816, State: Running, Role: LEADER
I20260929 10:50:03.470656 22431 consensus_queue.cc:237] T 64bd5b74f62941158bf836e4ae528807 P a1d4c6dbe57b4d4a867fd490b4926816 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } }
I20260929 10:50:03.471009 22471 tablet_replica.cc:439] T 64bd5b74f62941158bf836e4ae528807 P a1d4c6dbe57b4d4a867fd490b4926816: TxnStatusTablet state changed. Reason: New leader a1d4c6dbe57b4d4a867fd490b4926816. Latest consensus state: current_term: 1 leader_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } } }
I20260929 10:50:03.471119 22471 tablet_replica.cc:442] T 64bd5b74f62941158bf836e4ae528807 P a1d4c6dbe57b4d4a867fd490b4926816: This TxnStatusTablet replica's current role is: LEADER
I20260929 10:50:03.471284 22473 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 10:50:03.471411 22473 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 10:50:03.471431 21918 catalog_manager.cc:5495] T 64bd5b74f62941158bf836e4ae528807 P a1d4c6dbe57b4d4a867fd490b4926816 reported cstate change: term changed from 0 to 1, leader changed from <none> to a1d4c6dbe57b4d4a867fd490b4926816 (127.20.80.193). New cstate: current_term: 1 leader_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } health_report { overall_health: UNKNOWN } } }
I20260929 10:50:03.505551 22463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:03.502397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:34123 (local address 127.0.0.1:50594)
0929 10:50:03.502504 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:03.502518 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:03.502576 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:03.502798 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:03.502799 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:03.502801 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:50:03.502861 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:03.502863 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.503800 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.503801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:03.504450 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:03.504452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.504936 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.504937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:03.505052 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:03.505067 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:50:03.505385 (+ 318us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:50:03.505395 (+ 10us) client_negotiation.cc:769] Sending connection context
0929 10:50:03.505443 (+ 48us) client_negotiation.cc:240] Negotiation successful
0929 10:50:03.505485 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260929 10:50:03.505550 22421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:03.502536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50594 (local address 127.20.80.193:34123)
0929 10:50:03.502607 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:03.502610 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:03.502620 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:50:03.502658 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:03.502659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:03.502682 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:03.502767 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:50:03.502936 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.503715 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.504529 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.504840 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.504916 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:03.505161 (+ 245us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:50:03.505288 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:50:03.505368 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:03.505444 (+ 76us) server_negotiation.cc:299] Negotiation successful
0929 10:50:03.505467 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 10:50:03.506964 22216 raft_consensus.cc:1261] T 64bd5b74f62941158bf836e4ae528807 P 634f61a9482d490c9a3e58ac5690f289 [term 1 FOLLOWER]: Refusing update from remote peer a1d4c6dbe57b4d4a867fd490b4926816: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:03.507051 22356 raft_consensus.cc:1261] T 64bd5b74f62941158bf836e4ae528807 P 8632466c8c1d4c3d8398cfb4cd113a97 [term 1 FOLLOWER]: Refusing update from remote peer a1d4c6dbe57b4d4a867fd490b4926816: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:03.507364 22431 consensus_queue.cc:1035] T 64bd5b74f62941158bf836e4ae528807 P a1d4c6dbe57b4d4a867fd490b4926816 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:50:03.507503 22471 consensus_queue.cc:1035] T 64bd5b74f62941158bf836e4ae528807 P a1d4c6dbe57b4d4a867fd490b4926816 [LEADER]: Connected to new peer: Peer: permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:50:03.508009 22433 tablet_replica.cc:439] T 64bd5b74f62941158bf836e4ae528807 P 634f61a9482d490c9a3e58ac5690f289: TxnStatusTablet state changed. Reason: New leader a1d4c6dbe57b4d4a867fd490b4926816. Latest consensus state: current_term: 1 leader_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } } }
I20260929 10:50:03.508131 22433 tablet_replica.cc:442] T 64bd5b74f62941158bf836e4ae528807 P 634f61a9482d490c9a3e58ac5690f289: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:50:03.508118 22443 tablet_replica.cc:439] T 64bd5b74f62941158bf836e4ae528807 P 8632466c8c1d4c3d8398cfb4cd113a97: TxnStatusTablet state changed. Reason: New leader a1d4c6dbe57b4d4a867fd490b4926816. Latest consensus state: current_term: 1 leader_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } } }
I20260929 10:50:03.508209 22443 tablet_replica.cc:442] T 64bd5b74f62941158bf836e4ae528807 P 8632466c8c1d4c3d8398cfb4cd113a97: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:50:03.509189 22431 tablet_replica.cc:439] T 64bd5b74f62941158bf836e4ae528807 P a1d4c6dbe57b4d4a867fd490b4926816: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } } }
I20260929 10:50:03.509276 22431 tablet_replica.cc:442] T 64bd5b74f62941158bf836e4ae528807 P a1d4c6dbe57b4d4a867fd490b4926816: This TxnStatusTablet replica's current role is: LEADER
I20260929 10:50:03.509697 22431 tablet_replica.cc:439] T 64bd5b74f62941158bf836e4ae528807 P a1d4c6dbe57b4d4a867fd490b4926816: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } } }
I20260929 10:50:03.509792 22431 tablet_replica.cc:442] T 64bd5b74f62941158bf836e4ae528807 P a1d4c6dbe57b4d4a867fd490b4926816: This TxnStatusTablet replica's current role is: LEADER
I20260929 10:50:03.509900 22433 tablet_replica.cc:439] T 64bd5b74f62941158bf836e4ae528807 P 634f61a9482d490c9a3e58ac5690f289: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } } }
I20260929 10:50:03.509986 22433 tablet_replica.cc:442] T 64bd5b74f62941158bf836e4ae528807 P 634f61a9482d490c9a3e58ac5690f289: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:50:03.510208 22476 mvcc.cc:205] Tried to move back new op lower bound from 7334621198362255360 to 7334621198216425472. Current Snapshot: MvccSnapshot[applied={T|T < 7334621198362255360 or (T in {7334621198362255360})}]
I20260929 10:50:03.511102 22443 tablet_replica.cc:439] T 64bd5b74f62941158bf836e4ae528807 P 8632466c8c1d4c3d8398cfb4cd113a97: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 } } peers { permanent_uuid: "634f61a9482d490c9a3e58ac5690f289" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34501 } } peers { permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 } } }
I20260929 10:50:03.511195 22443 tablet_replica.cc:442] T 64bd5b74f62941158bf836e4ae528807 P 8632466c8c1d4c3d8398cfb4cd113a97: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:50:03.514290 22469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:03.511729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38472 (local address 127.20.80.194:34501)
0929 10:50:03.511770 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:03.511772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:03.511808 (+ 36us) server_negotiation.cc:407] Connection header received
0929 10:50:03.511892 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:03.511893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:03.511912 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:03.511979 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:50:03.512106 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.512591 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.513461 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.513670 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.513768 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:03.513930 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:50:03.514037 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:50:03.514117 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:03.514175 (+ 58us) server_negotiation.cc:299] Negotiation successful
0929 10:50:03.514197 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 10:50:03.519480 22488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:03.516754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:34123 (local address 127.20.80.194:46341)
0929 10:50:03.516889 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:03.516899 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:03.516959 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:03.517147 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:03.517147 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:03.517149 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:50:03.517191 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:03.517193 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.518050 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.518051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:03.518630 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:03.518632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.519096 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.519097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:03.519153 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:03.519166 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:50:03.519340 (+ 174us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:50:03.519341 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 10:50:03.519375 (+ 34us) client_negotiation.cc:240] Negotiation successful
0929 10:50:03.519411 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":73,"threads_started":1}
I20260929 10:50:03.519477 22421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:03.516819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:46341 (local address 127.20.80.193:34123)
0929 10:50:03.516880 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:03.516882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:03.516948 (+ 66us) server_negotiation.cc:407] Connection header received
0929 10:50:03.517071 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:03.517071 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:03.517083 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:03.517138 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:50:03.517232 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.517965 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.518742 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.519017 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.519088 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:03.519246 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:50:03.519282 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:50:03.519335 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:03.519379 (+ 44us) server_negotiation.cc:299] Negotiation successful
0929 10:50:03.519398 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 10:50:03.522039 22356 raft_consensus.cc:1261] T a29139164a334d5a987efbb1e1c50054 P 8632466c8c1d4c3d8398cfb4cd113a97 [term 1 FOLLOWER]: Refusing update from remote peer 634f61a9482d490c9a3e58ac5690f289: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:03.522104 22076 raft_consensus.cc:1261] T a29139164a334d5a987efbb1e1c50054 P a1d4c6dbe57b4d4a867fd490b4926816 [term 1 FOLLOWER]: Refusing update from remote peer 634f61a9482d490c9a3e58ac5690f289: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:03.522300 22433 consensus_queue.cc:1035] T a29139164a334d5a987efbb1e1c50054 P 634f61a9482d490c9a3e58ac5690f289 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8632466c8c1d4c3d8398cfb4cd113a97" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:50:03.522481 22456 consensus_queue.cc:1035] T a29139164a334d5a987efbb1e1c50054 P 634f61a9482d490c9a3e58ac5690f289 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1d4c6dbe57b4d4a867fd490b4926816" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:50:03.523988 22479 mvcc.cc:205] Tried to move back new op lower bound from 7334621198424190976 to 7334621197443342336. Current Snapshot: MvccSnapshot[applied={T|T < 7334621198424190976}]
I20260929 10:50:03.532140 22469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:03.529460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:36549 (local address 127.20.80.194:34501)
0929 10:50:03.529505 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:03.529507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:03.529599 (+ 92us) server_negotiation.cc:407] Connection header received
0929 10:50:03.529676 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:03.529676 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:03.529686 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:03.529732 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:50:03.529858 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.530373 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.531434 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.531632 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.531683 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:03.531905 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:50:03.531933 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:50:03.532004 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:03.532056 (+ 52us) server_negotiation.cc:299] Negotiation successful
0929 10:50:03.532077 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 10:50:03.532178 22494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:03.529379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:34501 (local address 127.20.80.193:36549)
0929 10:50:03.529539 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:03.529552 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:03.529617 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:03.529735 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:03.529736 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:03.529738 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:50:03.529803 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:03.529805 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.530454 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.530455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:03.531351 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:03.531354 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.531734 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.531735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:03.531807 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:03.531824 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:50:03.532006 (+ 182us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:50:03.532007 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 10:50:03.532048 (+ 41us) client_negotiation.cc:240] Negotiation successful
0929 10:50:03.532092 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":95,"threads_started":1}
I20260929 10:50:03.541507 21919 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:50:03.541625 21919 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:50:03.542299 21919 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2d9860e82bad4dda902f038e378b7410: Sending DeleteTablet for 3 replicas of tablet a29139164a334d5a987efbb1e1c50054
I20260929 10:50:03.542568 22056 tablet_service.cc:1514] Processing DeleteTablet for tablet a29139164a334d5a987efbb1e1c50054 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:03 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:52302
I20260929 10:50:03.542646 22336 tablet_service.cc:1514] Processing DeleteTablet for tablet a29139164a334d5a987efbb1e1c50054 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:03 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:47516
I20260929 10:50:03.542901 22497 tablet_replica.cc:330] T a29139164a334d5a987efbb1e1c50054 P a1d4c6dbe57b4d4a867fd490b4926816: stopping tablet replica
I20260929 10:50:03.542994 22497 raft_consensus.cc:2229] T a29139164a334d5a987efbb1e1c50054 P a1d4c6dbe57b4d4a867fd490b4926816 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:50:03.543006 22196 tablet_service.cc:1514] Processing DeleteTablet for tablet a29139164a334d5a987efbb1e1c50054 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:03 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:43034
I20260929 10:50:03.543124 22497 raft_consensus.cc:2258] T a29139164a334d5a987efbb1e1c50054 P a1d4c6dbe57b4d4a867fd490b4926816 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:03.543437 22497 ts_tablet_manager.cc:1905] T a29139164a334d5a987efbb1e1c50054 P a1d4c6dbe57b4d4a867fd490b4926816: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:03.543787 22498 tablet_replica.cc:330] T a29139164a334d5a987efbb1e1c50054 P 8632466c8c1d4c3d8398cfb4cd113a97: stopping tablet replica
I20260929 10:50:03.543859 22498 raft_consensus.cc:2229] T a29139164a334d5a987efbb1e1c50054 P 8632466c8c1d4c3d8398cfb4cd113a97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:50:03.543903 22498 raft_consensus.cc:2258] T a29139164a334d5a987efbb1e1c50054 P 8632466c8c1d4c3d8398cfb4cd113a97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:03.544030 22499 tablet_replica.cc:330] T a29139164a334d5a987efbb1e1c50054 P 634f61a9482d490c9a3e58ac5690f289: stopping tablet replica
I20260929 10:50:03.544085 22499 raft_consensus.cc:2229] T a29139164a334d5a987efbb1e1c50054 P 634f61a9482d490c9a3e58ac5690f289 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:50:03.544165 22499 raft_consensus.cc:2258] T a29139164a334d5a987efbb1e1c50054 P 634f61a9482d490c9a3e58ac5690f289 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:03.545732 22497 ts_tablet_manager.cc:1918] T a29139164a334d5a987efbb1e1c50054 P a1d4c6dbe57b4d4a867fd490b4926816: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:50:03.545840 22497 log.cc:1196] T a29139164a334d5a987efbb1e1c50054 P a1d4c6dbe57b4d4a867fd490b4926816: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-0/wal/wals/a29139164a334d5a987efbb1e1c50054
I20260929 10:50:03.546146 22497 ts_tablet_manager.cc:1939] T a29139164a334d5a987efbb1e1c50054 P a1d4c6dbe57b4d4a867fd490b4926816: Deleting consensus metadata
I20260929 10:50:03.548915 21905 catalog_manager.cc:4841] TS a1d4c6dbe57b4d4a867fd490b4926816 (127.20.80.193:34123): tablet a29139164a334d5a987efbb1e1c50054 (table test-table [id=39fed141b3df4cae9aa71bb16ded6922]) successfully deleted
I20260929 10:50:03.549441 22498 ts_tablet_manager.cc:1905] T a29139164a334d5a987efbb1e1c50054 P 8632466c8c1d4c3d8398cfb4cd113a97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:03.551376 22499 ts_tablet_manager.cc:1905] T a29139164a334d5a987efbb1e1c50054 P 634f61a9482d490c9a3e58ac5690f289: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:03.553527 22498 ts_tablet_manager.cc:1918] T a29139164a334d5a987efbb1e1c50054 P 8632466c8c1d4c3d8398cfb4cd113a97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:50:03.553596 22498 log.cc:1196] T a29139164a334d5a987efbb1e1c50054 P 8632466c8c1d4c3d8398cfb4cd113a97: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-2/wal/wals/a29139164a334d5a987efbb1e1c50054
I20260929 10:50:03.553853 22498 ts_tablet_manager.cc:1939] T a29139164a334d5a987efbb1e1c50054 P 8632466c8c1d4c3d8398cfb4cd113a97: Deleting consensus metadata
I20260929 10:50:03.554188 21904 catalog_manager.cc:4841] TS 8632466c8c1d4c3d8398cfb4cd113a97 (127.20.80.195:40015): tablet a29139164a334d5a987efbb1e1c50054 (table test-table [id=39fed141b3df4cae9aa71bb16ded6922]) successfully deleted
I20260929 10:50:03.554294 22499 ts_tablet_manager.cc:1918] T a29139164a334d5a987efbb1e1c50054 P 634f61a9482d490c9a3e58ac5690f289: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:50:03.554358 22499 log.cc:1196] T a29139164a334d5a987efbb1e1c50054 P 634f61a9482d490c9a3e58ac5690f289: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790678996126930-20803-0/minicluster-data/ts-1/wal/wals/a29139164a334d5a987efbb1e1c50054
I20260929 10:50:03.554608 22499 ts_tablet_manager.cc:1939] T a29139164a334d5a987efbb1e1c50054 P 634f61a9482d490c9a3e58ac5690f289: Deleting consensus metadata
I20260929 10:50:03.554932 21904 catalog_manager.cc:4841] TS 634f61a9482d490c9a3e58ac5690f289 (127.20.80.194:34501): tablet a29139164a334d5a987efbb1e1c50054 (table test-table [id=39fed141b3df4cae9aa71bb16ded6922]) successfully deleted
Sep 29 10:50:03 dist-test-slave-vqch krb5kdc[21874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679002, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.80.193@KRBTEST.COM
I20260929 10:50:03.560770 22421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:03.548756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50598 (local address 127.20.80.193:34123)
0929 10:50:03.548864 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:03.548866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:03.549311 (+ 445us) server_negotiation.cc:407] Connection header received
0929 10:50:03.549413 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:03.549414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:03.549424 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:03.549454 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:03.550313 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.551054 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.552361 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.552718 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.552763 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:03.557137 (+ 4374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:03.557146 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:03.557146 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:03.557154 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:03.559277 (+ 2123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:03.559791 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:03.559794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:03.559795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:03.559862 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:03.560148 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:03.560149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:03.560149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:03.560311 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:03.560457 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:03.560567 (+ 110us) server_negotiation.cc:299] Negotiation successful
0929 10:50:03.560648 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
W20260929 10:50:03.561034 22116 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50598
I20260929 10:50:03.572899 22464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:03.562020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53538 (local address 127.20.80.254:42475)
0929 10:50:03.562070 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:03.562072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:03.565169 (+ 3097us) server_negotiation.cc:407] Connection header received
0929 10:50:03.565226 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:03.565228 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:03.565245 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:03.565311 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:03.566007 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.566811 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.567834 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:03.568161 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:03.568232 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:03.569863 (+ 1631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:03.569865 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:03.569865 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:03.569873 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:03.571593 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:03.572009 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:03.572010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:03.572011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:03.572073 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:03.572304 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:03.572305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:03.572305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:03.572455 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:03.572571 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:03.572709 (+ 138us) server_negotiation.cc:299] Negotiation successful
0929 10:50:03.572799 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260929 10:50:03.573264 21919 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53538
I20260929 10:50:03.574011 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 21991
I20260929 10:50:03.579914 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 22131
I20260929 10:50:03.584897 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 22271
I20260929 10:50:03.590801 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 21887
2026-09-29T10:50:03Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2743 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:50:03 dist-test-slave-vqch krb5kdc[22506](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:50:03 dist-test-slave-vqch krb5kdc[22506](info): set up 2 sockets
Sep 29 10:50:03 dist-test-slave-vqch krb5kdc[22506](info): commencing operation
krb5kdc: starting...
W20260929 10:50:04.660881 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:50:04 dist-test-slave-vqch krb5kdc[22506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:50:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:50:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.80.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
I20260929 10:50:04.780459 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:43621
--webserver_interface=127.20.80.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:44913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:43621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:04.861564 22519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:04.861807 22519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:04.861845 22519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:04.863009 22519 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:50:04.863075 22519 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 10:50:04.863097 22519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:04.863119 22519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:04.863147 22519 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:50:04.863169 22519 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:50:04.864701 22519 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:44913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:43621
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:43621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:04.865002 22519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:04.865242 22519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:04.870282 22524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:04.870386 22525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:04.870415 22519 server_base.cc:1034] running on GCE node
W20260929 10:50:04.870309 22527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:04.870779 22519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:04.871078 22519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:04.872267 22519 hybrid_clock.cc:648] HybridClock initialized: now 1790679004872264 us; error 42 us; skew 500 ppm
Sep 29 10:50:04 dist-test-slave-vqch krb5kdc[22506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679004, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:04.874835 22519 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:50:04.875320 22519 webserver.cc:460] Webserver started at http://127.20.80.254:42291/ using document root <none> and password file <none>
I20260929 10:50:04.875557 22519 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:04.875624 22519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:04.875770 22519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:04.876844 22519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "0c45c4ef55f34833825183d938c0215c"
format_stamp: "Formatted at 2026-09-29 10:50:04 on dist-test-slave-vqch"
server_key: "cc9550a4a24c4a019ef65936adc13187"
server_key_iv: "342bd985114bd7ff3cee1093537ce08f"
server_key_version: "encryptionkey@0"
I20260929 10:50:04.877242 22519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "0c45c4ef55f34833825183d938c0215c"
format_stamp: "Formatted at 2026-09-29 10:50:04 on dist-test-slave-vqch"
server_key: "cc9550a4a24c4a019ef65936adc13187"
server_key_iv: "342bd985114bd7ff3cee1093537ce08f"
server_key_version: "encryptionkey@0"
I20260929 10:50:04.878629 22519 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:04.879426 22534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:04.879730 22519 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:04.879845 22519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "0c45c4ef55f34833825183d938c0215c"
format_stamp: "Formatted at 2026-09-29 10:50:04 on dist-test-slave-vqch"
server_key: "cc9550a4a24c4a019ef65936adc13187"
server_key_iv: "342bd985114bd7ff3cee1093537ce08f"
server_key_version: "encryptionkey@0"
I20260929 10:50:04.879922 22519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:04.908640 22519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:04.910759 22519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:04.910924 22519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:04.915715 22519 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:43621
I20260929 10:50:04.915774 22596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:43621 every 8 connection(s)
I20260929 10:50:04.916105 22519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:50:04.916702 22597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:04.919011 22597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c45c4ef55f34833825183d938c0215c: Bootstrap starting.
I20260929 10:50:04.919601 22597 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c45c4ef55f34833825183d938c0215c: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:04.919900 22597 log.cc:824] T 00000000000000000000000000000000 P 0c45c4ef55f34833825183d938c0215c: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:04.920593 22597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c45c4ef55f34833825183d938c0215c: No bootstrap required, opened a new log
I20260929 10:50:04.922780 22597 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c45c4ef55f34833825183d938c0215c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c45c4ef55f34833825183d938c0215c" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 43621 } }
I20260929 10:50:04.922953 22597 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c45c4ef55f34833825183d938c0215c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:04.922999 22597 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c45c4ef55f34833825183d938c0215c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c45c4ef55f34833825183d938c0215c, State: Initialized, Role: FOLLOWER
I20260929 10:50:04.923197 22597 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c45c4ef55f34833825183d938c0215c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c45c4ef55f34833825183d938c0215c" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 43621 } }
I20260929 10:50:04.923333 22600 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c45c4ef55f34833825183d938c0215c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:50:04.923421 22600 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c45c4ef55f34833825183d938c0215c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c45c4ef55f34833825183d938c0215c" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 43621 } }
I20260929 10:50:04.923286 22597 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c45c4ef55f34833825183d938c0215c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:50:04.923560 22600 leader_election.cc:302] T 00000000000000000000000000000000 P 0c45c4ef55f34833825183d938c0215c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c45c4ef55f34833825183d938c0215c; no voters:
I20260929 10:50:04.923604 22597 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c45c4ef55f34833825183d938c0215c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:50:04.923650 22597 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c45c4ef55f34833825183d938c0215c [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:04.923763 22600 leader_election.cc:288] T 00000000000000000000000000000000 P 0c45c4ef55f34833825183d938c0215c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260929 10:50:04.924274 22597 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c45c4ef55f34833825183d938c0215c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c45c4ef55f34833825183d938c0215c" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 43621 } }
I20260929 10:50:04.924367 22597 leader_election.cc:302] T 00000000000000000000000000000000 P 0c45c4ef55f34833825183d938c0215c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c45c4ef55f34833825183d938c0215c; no voters:
I20260929 10:50:04.924386 22601 raft_consensus.cc:2741] T 00000000000000000000000000000000 P 0c45c4ef55f34833825183d938c0215c [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260929 10:50:04.924410 22597 leader_election.cc:288] T 00000000000000000000000000000000 P 0c45c4ef55f34833825183d938c0215c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:50:04.924567 22600 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c45c4ef55f34833825183d938c0215c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:04.924676 22600 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c45c4ef55f34833825183d938c0215c [term 1 LEADER]: Becoming Leader. State: Replica: 0c45c4ef55f34833825183d938c0215c, State: Running, Role: LEADER
I20260929 10:50:04.924753 22597 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c45c4ef55f34833825183d938c0215c [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:50:04.924777 22600 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c45c4ef55f34833825183d938c0215c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c45c4ef55f34833825183d938c0215c" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 43621 } }
I20260929 10:50:04.925181 22601 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c45c4ef55f34833825183d938c0215c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c45c4ef55f34833825183d938c0215c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c45c4ef55f34833825183d938c0215c" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 43621 } } }
I20260929 10:50:04.925463 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 22519
I20260929 10:50:04.925489 22601 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c45c4ef55f34833825183d938c0215c [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:04.925554 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:50:04.925719 20803 external_mini_cluster.cc:1428] Setting key e6bf7a8e8866602bb4dc731c87eb1bad
I20260929 10:50:04.925405 22602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c45c4ef55f34833825183d938c0215c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c45c4ef55f34833825183d938c0215c. Latest consensus state: current_term: 1 leader_uuid: "0c45c4ef55f34833825183d938c0215c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c45c4ef55f34833825183d938c0215c" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 43621 } } }
I20260929 10:50:04.925812 22602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c45c4ef55f34833825183d938c0215c [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:04.925966 22609 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:50:04.926750 22609 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:50:04.929347 22609 catalog_manager.cc:1347] Generated new cluster ID: 028f7ae89e3845bf8d7be5f39311db2c
I20260929 10:50:04.929404 22609 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 10:50:04 dist-test-slave-vqch krb5kdc[22506](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:04.937170 22609 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:50:04.937888 22618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:04.926378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52476 (local address 127.20.80.254:43621)
0929 10:50:04.926536 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:04.926548 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:04.926616 (+ 68us) server_negotiation.cc:407] Connection header received
0929 10:50:04.926796 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:04.926804 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:04.926867 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:04.927806 (+ 939us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:04.927899 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:04.929327 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:04.930427 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:04.931049 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:04.931484 (+ 435us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:04.934035 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:04.934039 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:04.934039 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:04.934051 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:04.936020 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:04.936442 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:04.936445 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:04.936445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:04.936553 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:04.936930 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:04.936932 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:04.936932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:04.937202 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:04.937385 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:04.937483 (+ 98us) server_negotiation.cc:299] Negotiation successful
0929 10:50:04.937630 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":57,"threads_started":1}
I20260929 10:50:04.938879 22609 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:50:04.944557 22609 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c45c4ef55f34833825183d938c0215c: Generated new TSK 0
I20260929 10:50:04.944726 22609 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.80.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
I20260929 10:50:04.972131 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.193:0
--local_ip_for_outbound_sockets=127.20.80.193
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:43621
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:44913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:05.076874 22623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:05.077168 22623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:05.077198 22623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:05.078388 22623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:05.078435 22623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:05.078456 22623 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 10:50:05.078469 22623 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 10:50:05.078506 22623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.193
I20260929 10:50:05.080024 22623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:44913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.80.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.80.254:43621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.193
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:05.080298 22623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:05.080534 22623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:05.085844 22631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:05.085908 22628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:05.085829 22629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:05.086469 22623 server_base.cc:1034] running on GCE node
I20260929 10:50:05.086731 22623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:05.087103 22623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:05.088296 22623 hybrid_clock.cc:648] HybridClock initialized: now 1790679005088273 us; error 40 us; skew 500 ppm
Sep 29 10:50:05 dist-test-slave-vqch krb5kdc[22506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679005, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:05.091293 22623 init.cc:377] Logged in from keytab as kudu/127.20.80.193@KRBTEST.COM (short username kudu)
I20260929 10:50:05.091825 22623 webserver.cc:460] Webserver started at http://127.20.80.193:32985/ using document root <none> and password file <none>
I20260929 10:50:05.092083 22623 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:05.092139 22623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:05.092275 22623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:05.093744 22623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-0/data/instance:
uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0"
format_stamp: "Formatted at 2026-09-29 10:50:05 on dist-test-slave-vqch"
server_key: "dff3bf8c813d73beec8092bb98f2fac4"
server_key_iv: "3799c94f93fd95c1bc0a8bfa6395678c"
server_key_version: "encryptionkey@0"
I20260929 10:50:05.094238 22623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance:
uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0"
format_stamp: "Formatted at 2026-09-29 10:50:05 on dist-test-slave-vqch"
server_key: "dff3bf8c813d73beec8092bb98f2fac4"
server_key_iv: "3799c94f93fd95c1bc0a8bfa6395678c"
server_key_version: "encryptionkey@0"
I20260929 10:50:05.095928 22623 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:50:05.096793 22638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:05.097016 22623 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:05.097110 22623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-0/wal
uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0"
format_stamp: "Formatted at 2026-09-29 10:50:05 on dist-test-slave-vqch"
server_key: "dff3bf8c813d73beec8092bb98f2fac4"
server_key_iv: "3799c94f93fd95c1bc0a8bfa6395678c"
server_key_version: "encryptionkey@0"
I20260929 10:50:05.097182 22623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:05.109232 22623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:05.111102 22623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:05.111224 22623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:05.111768 22623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:05.111809 22623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:05.111833 22623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:05.111878 22623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 10:50:05 dist-test-slave-vqch krb5kdc[22506](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679005, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:05.119038 22623 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.193:34995
I20260929 10:50:05.119118 22753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.193:34995 every 8 connection(s)
I20260929 10:50:05.119445 22623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
I20260929 10:50:05.122169 22618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.112260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:33713 (local address 127.20.80.254:43621)
0929 10:50:05.112339 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:05.112342 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:05.112544 (+ 202us) server_negotiation.cc:407] Connection header received
0929 10:50:05.113123 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:05.113124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:05.113142 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:05.113206 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:05.114109 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.114959 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.116524 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.116869 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.116958 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.119235 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:05.119237 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:05.119237 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:05.119246 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:05.120822 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:05.121291 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:05.121292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:05.121292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:05.121346 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:05.121557 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:05.121557 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:05.121558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:05.121664 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:05.121713 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:05.122020 (+ 307us) server_negotiation.cc:299] Negotiation successful
0929 10:50:05.122103 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 10:50:05.122328 22647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.112215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:43621 (local address 127.20.80.193:33713)
0929 10:50:05.112404 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:05.112448 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:05.112958 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:05.113298 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:05.113302 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:05.113687 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:05.113989 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:05.113996 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.115056 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.115057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:05.116428 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:05.116430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.116962 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.116963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:05.117139 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.117684 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:05.117693 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:05.119120 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:05.120902 (+ 1782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:05.120909 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:05.120911 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:05.121223 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:05.121406 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:05.121406 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:05.121407 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:05.121504 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:05.121764 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:05.121771 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:05.121897 (+ 126us) client_negotiation.cc:769] Sending connection context
0929 10:50:05.121984 (+ 87us) client_negotiation.cc:240] Negotiation successful
0929 10:50:05.122111 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":91,"threads_started":1}
I20260929 10:50:05.122879 22754 heartbeater.cc:347] Connected to a master server at 127.20.80.254:43621
I20260929 10:50:05.123001 22754 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:05.123178 22754 heartbeater.cc:511] Master 127.20.80.254:43621 requested a full tablet report, sending...
I20260929 10:50:05.123776 22550 ts_manager.cc:194] Registered new tserver with Master: ec5e0cf1f22e44fba4bd414b63c9cad0 (127.20.80.193:34995)
I20260929 10:50:05.124703 22550 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.193@KRBTEST.COM'} at 127.20.80.193:33713
I20260929 10:50:05.127828 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 22623
I20260929 10:50:05.127962 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance
I20260929 10:50:05.128154 20803 external_mini_cluster.cc:1428] Setting key f5d995a6ab175994c6aab891b2d8d0ee
I20260929 10:50:05.130630 22618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.124010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:48789 (local address 127.20.80.254:43621)
0929 10:50:05.124045 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:05.124047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:05.124091 (+ 44us) server_negotiation.cc:407] Connection header received
0929 10:50:05.124159 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:05.124159 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:05.124167 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:05.124213 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:05.124535 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.125252 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.126289 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.126650 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.126721 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.127560 (+ 839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:05.127561 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:05.127561 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:05.127568 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:05.129373 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:05.129727 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:05.129728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:05.129728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:05.129792 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:05.130014 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:05.130015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:05.130015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:05.130156 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:05.130235 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:05.130453 (+ 218us) server_negotiation.cc:299] Negotiation successful
0929 10:50:05.130538 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260929 10:50:05.130651 22760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.123899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:43621 (local address 127.20.80.193:48789)
0929 10:50:05.124037 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:05.124047 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:05.124105 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:05.124219 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:05.124219 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:05.124405 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:05.124475 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:05.124476 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.125337 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.125338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:05.126201 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:05.126203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.126734 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.126735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:05.126813 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.127179 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:05.127183 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:05.127488 (+ 305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:05.129466 (+ 1978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:05.129467 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:05.129467 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:05.129653 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:05.129874 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:05.129875 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:05.129875 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:05.129938 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:05.130251 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:05.130252 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:05.130309 (+ 57us) client_negotiation.cc:769] Sending connection context
0929 10:50:05.130422 (+ 113us) client_negotiation.cc:240] Negotiation successful
0929 10:50:05.130536 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":7,"thread_start_us":74,"threads_started":1}
WARNING: no policy specified for kudu/127.20.80.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
I20260929 10:50:05.156330 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.194:0
--local_ip_for_outbound_sockets=127.20.80.194
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:43621
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:44913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:05.239813 22763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:05.240058 22763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:05.240099 22763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:05.241294 22763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:05.241364 22763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:05.241400 22763 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 10:50:05.241430 22763 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 10:50:05.241467 22763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.194
I20260929 10:50:05.242955 22763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:44913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.80.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.80.254:43621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.194
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:05.243223 22763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:05.243455 22763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:05.248677 22771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:05.248693 22769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:05.248687 22768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:05.248739 22763 server_base.cc:1034] running on GCE node
I20260929 10:50:05.249083 22763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:05.249287 22763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:05.250430 22763 hybrid_clock.cc:648] HybridClock initialized: now 1790679005250416 us; error 29 us; skew 500 ppm
Sep 29 10:50:05 dist-test-slave-vqch krb5kdc[22506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679005, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:05.253010 22763 init.cc:377] Logged in from keytab as kudu/127.20.80.194@KRBTEST.COM (short username kudu)
I20260929 10:50:05.253441 22763 webserver.cc:460] Webserver started at http://127.20.80.194:45071/ using document root <none> and password file <none>
I20260929 10:50:05.253669 22763 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:05.253736 22763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:05.253877 22763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:05.254942 22763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-1/data/instance:
uuid: "bb91879b1a8d4c38b9af948109cafe10"
format_stamp: "Formatted at 2026-09-29 10:50:05 on dist-test-slave-vqch"
server_key: "462e31481a2024aa3f60cc303458b2f2"
server_key_iv: "37fe3479720e8b3353c3697998d9ada0"
server_key_version: "encryptionkey@0"
I20260929 10:50:05.255306 22763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance:
uuid: "bb91879b1a8d4c38b9af948109cafe10"
format_stamp: "Formatted at 2026-09-29 10:50:05 on dist-test-slave-vqch"
server_key: "462e31481a2024aa3f60cc303458b2f2"
server_key_iv: "37fe3479720e8b3353c3697998d9ada0"
server_key_version: "encryptionkey@0"
I20260929 10:50:05.256688 22763 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:50:05.257534 22778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:05.257797 22763 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 10:50:05.257899 22763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-1/wal
uuid: "bb91879b1a8d4c38b9af948109cafe10"
format_stamp: "Formatted at 2026-09-29 10:50:05 on dist-test-slave-vqch"
server_key: "462e31481a2024aa3f60cc303458b2f2"
server_key_iv: "37fe3479720e8b3353c3697998d9ada0"
server_key_version: "encryptionkey@0"
I20260929 10:50:05.257979 22763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:05.280300 22763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:05.282841 22763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:05.283007 22763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:05.283535 22763 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:05.283569 22763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:05.283617 22763 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:05.283645 22763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:05.290474 22763 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.194:35241
I20260929 10:50:05.290552 22893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.194:35241 every 8 connection(s)
I20260929 10:50:05.290974 22763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
Sep 29 10:50:05 dist-test-slave-vqch krb5kdc[22506](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679005, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:05.291560 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 22763
I20260929 10:50:05.291664 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance
I20260929 10:50:05.291894 20803 external_mini_cluster.cc:1428] Setting key 6c041b62300a0e80154ae61a1e7298d8
I20260929 10:50:05.294807 22618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.284109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:49947 (local address 127.20.80.254:43621)
0929 10:50:05.284163 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:05.284166 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:05.284472 (+ 306us) server_negotiation.cc:407] Connection header received
0929 10:50:05.285011 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:05.285012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:05.285030 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:05.285099 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:05.285918 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.286769 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.288394 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.288744 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.288818 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.291602 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:05.291604 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:05.291604 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:05.291613 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:05.293402 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:05.293782 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:05.293783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:05.293783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:05.293848 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:05.294094 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:05.294096 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:05.294096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:05.294225 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:05.294302 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:05.294578 (+ 276us) server_negotiation.cc:299] Negotiation successful
0929 10:50:05.294667 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260929 10:50:05.294989 22787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.284124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:43621 (local address 127.20.80.194:49947)
0929 10:50:05.284312 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:05.284348 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:05.284870 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:05.285182 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:05.285186 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:05.285518 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:05.285808 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:05.285815 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.286882 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.286883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:05.288257 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:05.288260 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.288827 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.288828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:05.289012 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.289624 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:05.289634 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:05.291502 (+ 1868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:05.293481 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:05.293487 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:05.293489 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:05.293710 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:05.293913 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:05.293914 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:05.293914 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:05.294010 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:05.294293 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:05.294300 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:05.294436 (+ 136us) client_negotiation.cc:769] Sending connection context
0929 10:50:05.294555 (+ 119us) client_negotiation.cc:240] Negotiation successful
0929 10:50:05.294723 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":113,"threads_started":1}
I20260929 10:50:05.295506 22894 heartbeater.cc:347] Connected to a master server at 127.20.80.254:43621
I20260929 10:50:05.295614 22894 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:05.295796 22894 heartbeater.cc:511] Master 127.20.80.254:43621 requested a full tablet report, sending...
I20260929 10:50:05.296191 22551 ts_manager.cc:194] Registered new tserver with Master: bb91879b1a8d4c38b9af948109cafe10 (127.20.80.194:35241)
I20260929 10:50:05.296669 22551 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.194@KRBTEST.COM'} at 127.20.80.194:49947
WARNING: no policy specified for kudu/127.20.80.195@KRBTEST.COM; defaulting to no policy
I20260929 10:50:05.302992 22618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.296801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:51775 (local address 127.20.80.254:43621)
0929 10:50:05.296873 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:05.296876 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:05.296884 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:50:05.296904 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:05.296904 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:05.296914 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:05.296971 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:05.297418 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.297931 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.299061 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.299239 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.299286 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.300368 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:05.300369 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:05.300369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:05.300376 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:05.301883 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:05.302200 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:05.302201 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:05.302201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:05.302249 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:05.302453 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:05.302454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:05.302454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:05.302601 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:05.302654 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:05.302849 (+ 195us) server_negotiation.cc:299] Negotiation successful
0929 10:50:05.302927 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 10:50:05.302999 22901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.296602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:43621 (local address 127.20.80.194:51775)
0929 10:50:05.296736 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:05.296748 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:05.296806 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:05.297015 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:05.297016 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:05.297254 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:05.297327 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:05.297329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.298033 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.298035 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:05.298975 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:05.298978 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.299307 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.299307 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:05.299394 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.299859 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:05.299865 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:05.300287 (+ 422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:05.301965 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:05.301966 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:05.301966 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:05.302133 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:05.302322 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:05.302323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:05.302323 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:05.302384 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:05.302692 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:05.302693 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:05.302752 (+ 59us) client_negotiation.cc:769] Sending connection context
0929 10:50:05.302832 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 10:50:05.302926 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":67,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
I20260929 10:50:05.320814 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.195:0
--local_ip_for_outbound_sockets=127.20.80.195
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:43621
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:44913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:05.404284 22903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:05.404551 22903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:05.404598 22903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:05.405880 22903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:05.405951 22903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:05.405987 22903 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 10:50:05.406013 22903 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 10:50:05.406054 22903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.195
I20260929 10:50:05.407594 22903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:44913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.80.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.80.254:43621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.195
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:05.407872 22903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:05.408114 22903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:05.413350 22909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:05.413498 22903 server_base.cc:1034] running on GCE node
W20260929 10:50:05.413354 22908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:05.413350 22911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:05.413806 22903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:05.414039 22903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:05.415179 22903 hybrid_clock.cc:648] HybridClock initialized: now 1790679005415167 us; error 43 us; skew 500 ppm
Sep 29 10:50:05 dist-test-slave-vqch krb5kdc[22506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679005, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:05.417821 22903 init.cc:377] Logged in from keytab as kudu/127.20.80.195@KRBTEST.COM (short username kudu)
I20260929 10:50:05.418335 22903 webserver.cc:460] Webserver started at http://127.20.80.195:35677/ using document root <none> and password file <none>
I20260929 10:50:05.418597 22903 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:05.418651 22903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:05.418829 22903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:05.419976 22903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-2/data/instance:
uuid: "9f1437b726864fe499094ac03563fe08"
format_stamp: "Formatted at 2026-09-29 10:50:05 on dist-test-slave-vqch"
server_key: "e19e22c7645d25520d78035822f901b1"
server_key_iv: "0288a80e4ae407df2d868370bbb44398"
server_key_version: "encryptionkey@0"
I20260929 10:50:05.420410 22903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance:
uuid: "9f1437b726864fe499094ac03563fe08"
format_stamp: "Formatted at 2026-09-29 10:50:05 on dist-test-slave-vqch"
server_key: "e19e22c7645d25520d78035822f901b1"
server_key_iv: "0288a80e4ae407df2d868370bbb44398"
server_key_version: "encryptionkey@0"
I20260929 10:50:05.421950 22903 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260929 10:50:05.422753 22918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:05.422945 22903 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:05.423044 22903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-2/wal
uuid: "9f1437b726864fe499094ac03563fe08"
format_stamp: "Formatted at 2026-09-29 10:50:05 on dist-test-slave-vqch"
server_key: "e19e22c7645d25520d78035822f901b1"
server_key_iv: "0288a80e4ae407df2d868370bbb44398"
server_key_version: "encryptionkey@0"
I20260929 10:50:05.423136 22903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:05.431475 22903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:05.433876 22903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:05.433997 22903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:05.434494 22903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:05.434532 22903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:05.434556 22903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:05.434568 22903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:05.441236 22903 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.195:41207
I20260929 10:50:05.441314 23033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.195:41207 every 8 connection(s)
I20260929 10:50:05.441690 22903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
Sep 29 10:50:05 dist-test-slave-vqch krb5kdc[22506](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679005, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:05.445931 22927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.435076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:43621 (local address 127.20.80.195:38979)
0929 10:50:05.435244 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:05.435285 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:05.435915 (+ 630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:05.436298 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:05.436304 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:05.436739 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:05.437090 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:05.437098 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.438184 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.438185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:05.439540 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:05.439543 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.440068 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.440069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:05.440262 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.440897 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:05.440908 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:05.442760 (+ 1852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:05.444403 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:05.444409 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:05.444411 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:05.444697 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:05.444997 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:05.444998 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:05.444998 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:05.445139 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:05.445434 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:05.445439 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:05.445558 (+ 119us) client_negotiation.cc:769] Sending connection context
0929 10:50:05.445638 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 10:50:05.445761 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":98,"threads_started":1}
I20260929 10:50:05.445927 22618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.435117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:38979 (local address 127.20.80.254:43621)
0929 10:50:05.435184 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:05.435187 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:05.435373 (+ 186us) server_negotiation.cc:407] Connection header received
0929 10:50:05.436046 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:05.436048 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:05.436063 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:05.436141 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:05.437205 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.438060 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.439634 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.439966 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.440047 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.442889 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:05.442890 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:05.442890 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:05.442899 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:05.444312 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:05.444789 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:05.444791 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:05.444792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:05.444863 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:05.445236 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:05.445237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:05.445238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:05.445369 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:05.445436 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:05.445680 (+ 244us) server_negotiation.cc:299] Negotiation successful
0929 10:50:05.445821 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 10:50:05.445922 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 22903
I20260929 10:50:05.446015 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance
I20260929 10:50:05.446170 20803 external_mini_cluster.cc:1428] Setting key cbb408ed4e770f782752297208d32b9b
I20260929 10:50:05.446609 23034 heartbeater.cc:347] Connected to a master server at 127.20.80.254:43621
I20260929 10:50:05.446725 23034 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:05.446934 23034 heartbeater.cc:511] Master 127.20.80.254:43621 requested a full tablet report, sending...
I20260929 10:50:05.447351 22549 ts_manager.cc:194] Registered new tserver with Master: 9f1437b726864fe499094ac03563fe08 (127.20.80.195:41207)
I20260929 10:50:05.447877 22549 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.195@KRBTEST.COM'} at 127.20.80.195:38979
I20260929 10:50:05.453506 23040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.447763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:43621 (local address 127.20.80.195:48675)
0929 10:50:05.447926 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:05.447935 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:05.448005 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:05.448126 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:05.448127 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:05.448306 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:05.448361 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:05.448363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.449031 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.449032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:05.449560 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:05.449562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.449946 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.449947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:05.450026 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.450456 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:05.450461 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:05.450774 (+ 313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:05.452394 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:05.452395 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:05.452396 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:05.452579 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:05.452796 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:05.452797 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:05.452797 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:05.452864 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:05.453152 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:05.453153 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:05.453203 (+ 50us) client_negotiation.cc:769] Sending connection context
0929 10:50:05.453288 (+ 85us) client_negotiation.cc:240] Negotiation successful
0929 10:50:05.453403 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":69,"threads_started":1}
I20260929 10:50:05.453502 22618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.447823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:48675 (local address 127.20.80.254:43621)
0929 10:50:05.447860 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:05.447862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:05.447989 (+ 127us) server_negotiation.cc:407] Connection header received
0929 10:50:05.448053 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:05.448054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:05.448064 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:05.448121 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:05.448432 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.448915 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.449645 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.449862 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.449944 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.450892 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:05.450893 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:05.450893 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:05.450901 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:05.452266 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:05.452664 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:05.452666 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:05.452666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:05.452730 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:05.452928 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:05.452928 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:05.452929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:05.453087 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:05.453163 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:05.453313 (+ 150us) server_negotiation.cc:299] Negotiation successful
0929 10:50:05.453400 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260929 10:50:05.458102 20803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 10:50:05 dist-test-slave-vqch krb5kdc[22506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 10:50:05 dist-test-slave-vqch krb5kdc[22506](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:05.482043 22618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.474464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52482 (local address 127.20.80.254:43621)
0929 10:50:05.474532 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:05.474534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:05.474542 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:50:05.474568 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:05.474569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:05.474578 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:05.474629 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:05.475298 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.475790 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.476504 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.476702 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.476764 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.479342 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:05.479343 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:05.479344 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:05.479351 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:05.480786 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:05.481241 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:05.481242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:05.481242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:05.481301 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:05.481496 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:05.481497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:05.481497 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:05.481612 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:05.481718 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:05.481824 (+ 106us) server_negotiation.cc:299] Negotiation successful
0929 10:50:05.481908 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 10:50:05.483675 22549 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52482:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:50:05.485286 22549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:50:05.491971 23053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.487671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:34995 (local address 127.0.0.1:35410)
0929 10:50:05.488053 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:05.488064 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:05.488108 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:05.488451 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:05.488452 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:05.488453 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:05.488488 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:05.488490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.489473 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.489474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:05.490708 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:05.490711 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.491653 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.491654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:05.491736 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.491755 (+ 19us) client_negotiation.cc:769] Sending connection context
0929 10:50:05.491820 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 10:50:05.491881 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"mutex_wait_us":5,"thread_start_us":57,"threads_started":1}
I20260929 10:50:05.491989 23054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.487881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35410 (local address 127.20.80.193:34995)
0929 10:50:05.488225 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:05.488247 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:05.488257 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:50:05.488294 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:05.488296 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:05.488313 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:05.488382 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:05.488557 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.489390 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.490812 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.491563 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.491646 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.491704 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:05.491852 (+ 148us) server_negotiation.cc:299] Negotiation successful
0929 10:50:05.491897 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":81,"threads_started":1}
I20260929 10:50:05.492081 23051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.487671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:35241 (local address 127.0.0.1:35996)
0929 10:50:05.487864 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:05.487906 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:05.487975 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:05.488398 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:05.488401 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:05.488403 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:05.488488 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:05.488493 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.489907 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.489909 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:05.490939 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:05.490941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.491835 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.491837 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:05.491928 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.491932 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:50:05.491963 (+ 31us) client_negotiation.cc:240] Negotiation successful
0929 10:50:05.492006 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":126,"threads_started":1}
I20260929 10:50:05.492897 23052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.487698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:41207 (local address 127.0.0.1:58792)
0929 10:50:05.488154 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:05.488164 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:05.488227 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:05.488532 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:05.488533 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:05.488535 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:05.488592 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:05.488593 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.491008 (+ 2415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.491009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:05.491705 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:05.491706 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.492644 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.492645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:05.492728 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.492733 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 10:50:05.492792 (+ 59us) client_negotiation.cc:240] Negotiation successful
0929 10:50:05.492836 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":117,"threads_started":1}
I20260929 10:50:05.492962 23055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.487946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58792 (local address 127.20.80.195:41207)
0929 10:50:05.488135 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:05.488140 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:05.488215 (+ 75us) server_negotiation.cc:407] Connection header received
0929 10:50:05.488359 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:05.488362 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:05.488378 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:05.488455 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:05.488680 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.489824 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.491789 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.492563 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.492634 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.492690 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:05.492816 (+ 126us) server_negotiation.cc:299] Negotiation successful
0929 10:50:05.492864 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":82,"threads_started":1}
I20260929 10:50:05.493191 22688 tablet_service.cc:1467] Processing CreateTablet for tablet 5865250ca4e5436f8f38121bb6711b1b (DEFAULT_TABLE table=test-table [id=f479876841b84b55b8e8f0d6367df2df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:05.493568 22688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5865250ca4e5436f8f38121bb6711b1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:05.493568 22968 tablet_service.cc:1467] Processing CreateTablet for tablet 5865250ca4e5436f8f38121bb6711b1b (DEFAULT_TABLE table=test-table [id=f479876841b84b55b8e8f0d6367df2df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:05.493906 22968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5865250ca4e5436f8f38121bb6711b1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:05.496490 23057 tablet_bootstrap.cc:492] T 5865250ca4e5436f8f38121bb6711b1b P 9f1437b726864fe499094ac03563fe08: Bootstrap starting.
I20260929 10:50:05.492417 23056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.487976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35996 (local address 127.20.80.194:35241)
0929 10:50:05.488140 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:05.488145 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:05.488155 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:50:05.488198 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:05.488201 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:05.488286 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:05.488567 (+ 281us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:05.488621 (+ 54us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.489817 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.491023 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.491739 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.492155 (+ 416us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.492221 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:05.492247 (+ 26us) server_negotiation.cc:299] Negotiation successful
0929 10:50:05.492311 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":82,"threads_started":1}
I20260929 10:50:05.497151 23058 tablet_bootstrap.cc:492] T 5865250ca4e5436f8f38121bb6711b1b P ec5e0cf1f22e44fba4bd414b63c9cad0: Bootstrap starting.
I20260929 10:50:05.497467 22828 tablet_service.cc:1467] Processing CreateTablet for tablet 5865250ca4e5436f8f38121bb6711b1b (DEFAULT_TABLE table=test-table [id=f479876841b84b55b8e8f0d6367df2df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:05.497563 23057 tablet_bootstrap.cc:654] T 5865250ca4e5436f8f38121bb6711b1b P 9f1437b726864fe499094ac03563fe08: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:05.497778 22828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5865250ca4e5436f8f38121bb6711b1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:05.497833 23058 tablet_bootstrap.cc:654] T 5865250ca4e5436f8f38121bb6711b1b P ec5e0cf1f22e44fba4bd414b63c9cad0: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:05.497902 23057 log.cc:824] T 5865250ca4e5436f8f38121bb6711b1b P 9f1437b726864fe499094ac03563fe08: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:05.498172 23058 log.cc:824] T 5865250ca4e5436f8f38121bb6711b1b P ec5e0cf1f22e44fba4bd414b63c9cad0: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:05.498667 23057 tablet_bootstrap.cc:492] T 5865250ca4e5436f8f38121bb6711b1b P 9f1437b726864fe499094ac03563fe08: No bootstrap required, opened a new log
I20260929 10:50:05.498723 23057 ts_tablet_manager.cc:1397] T 5865250ca4e5436f8f38121bb6711b1b P 9f1437b726864fe499094ac03563fe08: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:50:05.500391 23057 raft_consensus.cc:348] T 5865250ca4e5436f8f38121bb6711b1b P 9f1437b726864fe499094ac03563fe08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:05.500522 23057 raft_consensus.cc:374] T 5865250ca4e5436f8f38121bb6711b1b P 9f1437b726864fe499094ac03563fe08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:05.500545 23057 raft_consensus.cc:729] T 5865250ca4e5436f8f38121bb6711b1b P 9f1437b726864fe499094ac03563fe08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1437b726864fe499094ac03563fe08, State: Initialized, Role: FOLLOWER
I20260929 10:50:05.500667 23057 consensus_queue.cc:260] T 5865250ca4e5436f8f38121bb6711b1b P 9f1437b726864fe499094ac03563fe08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:05.500937 23057 ts_tablet_manager.cc:1428] T 5865250ca4e5436f8f38121bb6711b1b P 9f1437b726864fe499094ac03563fe08: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:50:05.501008 23034 heartbeater.cc:503] Master 127.20.80.254:43621 was elected leader, sending a full tablet report...
I20260929 10:50:05.501324 23062 tablet_bootstrap.cc:492] T 5865250ca4e5436f8f38121bb6711b1b P bb91879b1a8d4c38b9af948109cafe10: Bootstrap starting.
I20260929 10:50:05.501636 23058 tablet_bootstrap.cc:492] T 5865250ca4e5436f8f38121bb6711b1b P ec5e0cf1f22e44fba4bd414b63c9cad0: No bootstrap required, opened a new log
I20260929 10:50:05.501721 23058 ts_tablet_manager.cc:1397] T 5865250ca4e5436f8f38121bb6711b1b P ec5e0cf1f22e44fba4bd414b63c9cad0: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260929 10:50:05.501974 23062 tablet_bootstrap.cc:654] T 5865250ca4e5436f8f38121bb6711b1b P bb91879b1a8d4c38b9af948109cafe10: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:05.502358 23062 log.cc:824] T 5865250ca4e5436f8f38121bb6711b1b P bb91879b1a8d4c38b9af948109cafe10: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:05.503183 23062 tablet_bootstrap.cc:492] T 5865250ca4e5436f8f38121bb6711b1b P bb91879b1a8d4c38b9af948109cafe10: No bootstrap required, opened a new log
I20260929 10:50:05.503261 23062 ts_tablet_manager.cc:1397] T 5865250ca4e5436f8f38121bb6711b1b P bb91879b1a8d4c38b9af948109cafe10: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:50:05.503460 23058 raft_consensus.cc:348] T 5865250ca4e5436f8f38121bb6711b1b P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:05.503600 23058 raft_consensus.cc:374] T 5865250ca4e5436f8f38121bb6711b1b P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:05.503644 23058 raft_consensus.cc:729] T 5865250ca4e5436f8f38121bb6711b1b P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec5e0cf1f22e44fba4bd414b63c9cad0, State: Initialized, Role: FOLLOWER
I20260929 10:50:05.503756 23058 consensus_queue.cc:260] T 5865250ca4e5436f8f38121bb6711b1b P ec5e0cf1f22e44fba4bd414b63c9cad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:05.504031 23058 ts_tablet_manager.cc:1428] T 5865250ca4e5436f8f38121bb6711b1b P ec5e0cf1f22e44fba4bd414b63c9cad0: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 10:50:05.504063 22754 heartbeater.cc:503] Master 127.20.80.254:43621 was elected leader, sending a full tablet report...
I20260929 10:50:05.505291 23062 raft_consensus.cc:348] T 5865250ca4e5436f8f38121bb6711b1b P bb91879b1a8d4c38b9af948109cafe10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:05.505409 23062 raft_consensus.cc:374] T 5865250ca4e5436f8f38121bb6711b1b P bb91879b1a8d4c38b9af948109cafe10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:05.505429 23062 raft_consensus.cc:729] T 5865250ca4e5436f8f38121bb6711b1b P bb91879b1a8d4c38b9af948109cafe10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb91879b1a8d4c38b9af948109cafe10, State: Initialized, Role: FOLLOWER
I20260929 10:50:05.505498 23062 consensus_queue.cc:260] T 5865250ca4e5436f8f38121bb6711b1b P bb91879b1a8d4c38b9af948109cafe10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:05.505679 23062 ts_tablet_manager.cc:1428] T 5865250ca4e5436f8f38121bb6711b1b P bb91879b1a8d4c38b9af948109cafe10: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:50:05.505739 22894 heartbeater.cc:503] Master 127.20.80.254:43621 was elected leader, sending a full tablet report...
W20260929 10:50:05.541549 22895 tablet.cc:2369] T 5865250ca4e5436f8f38121bb6711b1b P bb91879b1a8d4c38b9af948109cafe10: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:50:05.620285 22755 tablet.cc:2369] T 5865250ca4e5436f8f38121bb6711b1b P ec5e0cf1f22e44fba4bd414b63c9cad0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:50:05.692348 23035 tablet.cc:2369] T 5865250ca4e5436f8f38121bb6711b1b P 9f1437b726864fe499094ac03563fe08: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:50:05.750712 23064 raft_consensus.cc:482] T 5865250ca4e5436f8f38121bb6711b1b P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:50:05.750849 23064 raft_consensus.cc:504] T 5865250ca4e5436f8f38121bb6711b1b P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:05.751283 23064 leader_election.cc:288] T 5865250ca4e5436f8f38121bb6711b1b P ec5e0cf1f22e44fba4bd414b63c9cad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb91879b1a8d4c38b9af948109cafe10 (127.20.80.194:35241), 9f1437b726864fe499094ac03563fe08 (127.20.80.195:41207)
I20260929 10:50:05.754356 23056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.751564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:37215 (local address 127.20.80.194:35241)
0929 10:50:05.751652 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:05.751655 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:05.751781 (+ 126us) server_negotiation.cc:407] Connection header received
0929 10:50:05.751847 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:05.751848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:05.751855 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:05.751924 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:05.752114 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.752573 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.753636 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.754014 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.754094 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.754103 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:05.754244 (+ 141us) server_negotiation.cc:299] Negotiation successful
0929 10:50:05.754282 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 10:50:05.754360 23055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.751621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:44483 (local address 127.20.80.195:41207)
0929 10:50:05.751667 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:05.751669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:05.751739 (+ 70us) server_negotiation.cc:407] Connection header received
0929 10:50:05.751831 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:05.751832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:05.751842 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:05.751895 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:05.752087 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.752575 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.753669 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.754014 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.754093 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.754103 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:05.754253 (+ 150us) server_negotiation.cc:299] Negotiation successful
0929 10:50:05.754289 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 10:50:05.754411 23066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.751481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:35241 (local address 127.20.80.193:37215)
0929 10:50:05.751672 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:05.751682 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:05.751779 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:05.751943 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:05.751944 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:05.751946 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:05.752032 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:05.752033 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.752659 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.752661 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:05.753546 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:05.753548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.754094 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.754095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:05.754171 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.754174 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:50:05.754239 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 10:50:05.754286 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":21,"thread_start_us":100,"threads_started":1}
I20260929 10:50:05.754411 23067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.751510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:41207 (local address 127.20.80.193:44483)
0929 10:50:05.751677 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:05.751686 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:05.751746 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:05.751915 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:05.751916 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:05.751919 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:05.752017 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:05.752018 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.752661 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.752662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:05.753558 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:05.753561 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.754096 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.754097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:05.754170 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.754173 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:50:05.754238 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 10:50:05.754282 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":55,"thread_start_us":86,"threads_started":1}
I20260929 10:50:05.754802 22988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5865250ca4e5436f8f38121bb6711b1b" candidate_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1437b726864fe499094ac03563fe08" is_pre_election: true
I20260929 10:50:05.754868 22848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5865250ca4e5436f8f38121bb6711b1b" candidate_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb91879b1a8d4c38b9af948109cafe10" is_pre_election: true
I20260929 10:50:05.754971 22988 raft_consensus.cc:2454] T 5865250ca4e5436f8f38121bb6711b1b P 9f1437b726864fe499094ac03563fe08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec5e0cf1f22e44fba4bd414b63c9cad0 in term 0.
I20260929 10:50:05.754984 22848 raft_consensus.cc:2454] T 5865250ca4e5436f8f38121bb6711b1b P bb91879b1a8d4c38b9af948109cafe10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec5e0cf1f22e44fba4bd414b63c9cad0 in term 0.
I20260929 10:50:05.755218 22639 leader_election.cc:302] T 5865250ca4e5436f8f38121bb6711b1b P ec5e0cf1f22e44fba4bd414b63c9cad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f1437b726864fe499094ac03563fe08, ec5e0cf1f22e44fba4bd414b63c9cad0; no voters:
I20260929 10:50:05.755391 23064 raft_consensus.cc:2781] T 5865250ca4e5436f8f38121bb6711b1b P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:50:05.755431 23064 raft_consensus.cc:482] T 5865250ca4e5436f8f38121bb6711b1b P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:50:05.755447 23064 raft_consensus.cc:3037] T 5865250ca4e5436f8f38121bb6711b1b P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:05.756181 23064 raft_consensus.cc:504] T 5865250ca4e5436f8f38121bb6711b1b P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:05.756331 23064 leader_election.cc:288] T 5865250ca4e5436f8f38121bb6711b1b P ec5e0cf1f22e44fba4bd414b63c9cad0 [CANDIDATE]: Term 1 election: Requested vote from peers bb91879b1a8d4c38b9af948109cafe10 (127.20.80.194:35241), 9f1437b726864fe499094ac03563fe08 (127.20.80.195:41207)
I20260929 10:50:05.756469 22848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5865250ca4e5436f8f38121bb6711b1b" candidate_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb91879b1a8d4c38b9af948109cafe10"
I20260929 10:50:05.756549 22848 raft_consensus.cc:3037] T 5865250ca4e5436f8f38121bb6711b1b P bb91879b1a8d4c38b9af948109cafe10 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:05.756559 22988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5865250ca4e5436f8f38121bb6711b1b" candidate_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1437b726864fe499094ac03563fe08"
I20260929 10:50:05.756645 22988 raft_consensus.cc:3037] T 5865250ca4e5436f8f38121bb6711b1b P 9f1437b726864fe499094ac03563fe08 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:05.757385 22848 raft_consensus.cc:2454] T 5865250ca4e5436f8f38121bb6711b1b P bb91879b1a8d4c38b9af948109cafe10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec5e0cf1f22e44fba4bd414b63c9cad0 in term 1.
I20260929 10:50:05.757514 22988 raft_consensus.cc:2454] T 5865250ca4e5436f8f38121bb6711b1b P 9f1437b726864fe499094ac03563fe08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec5e0cf1f22e44fba4bd414b63c9cad0 in term 1.
I20260929 10:50:05.757580 22640 leader_election.cc:302] T 5865250ca4e5436f8f38121bb6711b1b P ec5e0cf1f22e44fba4bd414b63c9cad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb91879b1a8d4c38b9af948109cafe10, ec5e0cf1f22e44fba4bd414b63c9cad0; no voters:
I20260929 10:50:05.757659 23064 raft_consensus.cc:2781] T 5865250ca4e5436f8f38121bb6711b1b P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:05.757755 23064 raft_consensus.cc:686] T 5865250ca4e5436f8f38121bb6711b1b P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 1 LEADER]: Becoming Leader. State: Replica: ec5e0cf1f22e44fba4bd414b63c9cad0, State: Running, Role: LEADER
I20260929 10:50:05.757861 23064 consensus_queue.cc:237] T 5865250ca4e5436f8f38121bb6711b1b P ec5e0cf1f22e44fba4bd414b63c9cad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:05.758683 22549 catalog_manager.cc:5495] T 5865250ca4e5436f8f38121bb6711b1b P ec5e0cf1f22e44fba4bd414b63c9cad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec5e0cf1f22e44fba4bd414b63c9cad0 (127.20.80.193). New cstate: current_term: 1 leader_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } health_report { overall_health: UNKNOWN } } }
Sep 29 10:50:05 dist-test-slave-vqch krb5kdc[22506](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679004, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:05.792428 23075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.785659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:43621 (local address 127.0.0.1:52492)
0929 10:50:05.785836 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:05.785849 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:05.785944 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:05.786100 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:05.786101 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:05.786418 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:05.786465 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:05.786468 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.787111 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.787112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:05.787651 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:05.787652 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.787985 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.787986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:05.788039 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.788561 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:05.788566 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:05.789955 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:05.791320 (+ 1365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:05.791326 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:05.791328 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:05.791523 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:05.791718 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:05.791719 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:05.791719 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:05.791784 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:05.792059 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:05.792065 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:05.792108 (+ 43us) client_negotiation.cc:769] Sending connection context
0929 10:50:05.792197 (+ 89us) client_negotiation.cc:240] Negotiation successful
0929 10:50:05.792294 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":85,"threads_started":1}
I20260929 10:50:05.792471 22618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.785780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52492 (local address 127.20.80.254:43621)
0929 10:50:05.785862 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:05.785865 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:05.785927 (+ 62us) server_negotiation.cc:407] Connection header received
0929 10:50:05.786023 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:05.786024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:05.786039 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:05.786096 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:05.786549 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.787032 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.787741 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.787916 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.787987 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.790058 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:05.790059 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:05.790059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:05.790066 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:05.791237 (+ 1171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:05.791592 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:05.791592 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:05.791593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:05.791642 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:05.791847 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:05.791848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:05.791848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:05.791994 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:05.792058 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:05.792216 (+ 158us) server_negotiation.cc:299] Negotiation successful
0929 10:50:05.792337 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":60}
I20260929 10:50:05.794545 22549 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:52492:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 10:50:05.794785 22549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:50:05.796348 22828 tablet_service.cc:1467] Processing CreateTablet for tablet a3a31565ff894c95b21198a7070b65ef (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=657b71c443464699861c7544a2ddfce8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 10:50:05.796336 22688 tablet_service.cc:1467] Processing CreateTablet for tablet a3a31565ff894c95b21198a7070b65ef (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=657b71c443464699861c7544a2ddfce8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 10:50:05.796391 22968 tablet_service.cc:1467] Processing CreateTablet for tablet a3a31565ff894c95b21198a7070b65ef (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=657b71c443464699861c7544a2ddfce8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 10:50:05.796519 22828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3a31565ff894c95b21198a7070b65ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:05.796519 22688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3a31565ff894c95b21198a7070b65ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:05.796527 22968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3a31565ff894c95b21198a7070b65ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:05.797775 23057 tablet_bootstrap.cc:492] T a3a31565ff894c95b21198a7070b65ef P 9f1437b726864fe499094ac03563fe08: Bootstrap starting.
I20260929 10:50:05.798058 23062 tablet_bootstrap.cc:492] T a3a31565ff894c95b21198a7070b65ef P bb91879b1a8d4c38b9af948109cafe10: Bootstrap starting.
I20260929 10:50:05.798391 23057 tablet_bootstrap.cc:654] T a3a31565ff894c95b21198a7070b65ef P 9f1437b726864fe499094ac03563fe08: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:05.798460 23062 tablet_bootstrap.cc:654] T a3a31565ff894c95b21198a7070b65ef P bb91879b1a8d4c38b9af948109cafe10: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:05.798785 23058 tablet_bootstrap.cc:492] T a3a31565ff894c95b21198a7070b65ef P ec5e0cf1f22e44fba4bd414b63c9cad0: Bootstrap starting.
I20260929 10:50:05.799125 23062 tablet_bootstrap.cc:492] T a3a31565ff894c95b21198a7070b65ef P bb91879b1a8d4c38b9af948109cafe10: No bootstrap required, opened a new log
I20260929 10:50:05.799185 23058 tablet_bootstrap.cc:654] T a3a31565ff894c95b21198a7070b65ef P ec5e0cf1f22e44fba4bd414b63c9cad0: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:05.799181 23062 ts_tablet_manager.cc:1397] T a3a31565ff894c95b21198a7070b65ef P bb91879b1a8d4c38b9af948109cafe10: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:05.799566 23062 raft_consensus.cc:348] T a3a31565ff894c95b21198a7070b65ef P bb91879b1a8d4c38b9af948109cafe10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:05.799654 23062 raft_consensus.cc:374] T a3a31565ff894c95b21198a7070b65ef P bb91879b1a8d4c38b9af948109cafe10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:05.799676 23062 raft_consensus.cc:729] T a3a31565ff894c95b21198a7070b65ef P bb91879b1a8d4c38b9af948109cafe10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb91879b1a8d4c38b9af948109cafe10, State: Initialized, Role: FOLLOWER
I20260929 10:50:05.799710 23062 consensus_queue.cc:260] T a3a31565ff894c95b21198a7070b65ef P bb91879b1a8d4c38b9af948109cafe10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:05.799823 23062 ts_tablet_manager.cc:1428] T a3a31565ff894c95b21198a7070b65ef P bb91879b1a8d4c38b9af948109cafe10: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260929 10:50:05.799887 23065 tablet_replica.cc:439] T a3a31565ff894c95b21198a7070b65ef P bb91879b1a8d4c38b9af948109cafe10: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } } }
I20260929 10:50:05.800004 23065 tablet_replica.cc:442] T a3a31565ff894c95b21198a7070b65ef P bb91879b1a8d4c38b9af948109cafe10: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:50:05.800032 23057 tablet_bootstrap.cc:492] T a3a31565ff894c95b21198a7070b65ef P 9f1437b726864fe499094ac03563fe08: No bootstrap required, opened a new log
I20260929 10:50:05.800076 23057 ts_tablet_manager.cc:1397] T a3a31565ff894c95b21198a7070b65ef P 9f1437b726864fe499094ac03563fe08: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:50:05.800241 23057 raft_consensus.cc:348] T a3a31565ff894c95b21198a7070b65ef P 9f1437b726864fe499094ac03563fe08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:05.800325 23058 tablet_bootstrap.cc:492] T a3a31565ff894c95b21198a7070b65ef P ec5e0cf1f22e44fba4bd414b63c9cad0: No bootstrap required, opened a new log
I20260929 10:50:05.800346 23057 raft_consensus.cc:374] T a3a31565ff894c95b21198a7070b65ef P 9f1437b726864fe499094ac03563fe08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:05.800371 23057 raft_consensus.cc:729] T a3a31565ff894c95b21198a7070b65ef P 9f1437b726864fe499094ac03563fe08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1437b726864fe499094ac03563fe08, State: Initialized, Role: FOLLOWER
I20260929 10:50:05.800364 23058 ts_tablet_manager.cc:1397] T a3a31565ff894c95b21198a7070b65ef P ec5e0cf1f22e44fba4bd414b63c9cad0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:50:05.800417 23057 consensus_queue.cc:260] T a3a31565ff894c95b21198a7070b65ef P 9f1437b726864fe499094ac03563fe08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:05.800518 23057 ts_tablet_manager.cc:1428] T a3a31565ff894c95b21198a7070b65ef P 9f1437b726864fe499094ac03563fe08: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:05.800504 23058 raft_consensus.cc:348] T a3a31565ff894c95b21198a7070b65ef P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:05.800568 23058 raft_consensus.cc:374] T a3a31565ff894c95b21198a7070b65ef P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:05.800606 23058 raft_consensus.cc:729] T a3a31565ff894c95b21198a7070b65ef P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec5e0cf1f22e44fba4bd414b63c9cad0, State: Initialized, Role: FOLLOWER
I20260929 10:50:05.800666 23058 consensus_queue.cc:260] T a3a31565ff894c95b21198a7070b65ef P ec5e0cf1f22e44fba4bd414b63c9cad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:05.800758 23058 ts_tablet_manager.cc:1428] T a3a31565ff894c95b21198a7070b65ef P ec5e0cf1f22e44fba4bd414b63c9cad0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:05.800796 23064 tablet_replica.cc:439] T a3a31565ff894c95b21198a7070b65ef P ec5e0cf1f22e44fba4bd414b63c9cad0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } } }
I20260929 10:50:05.800868 23064 tablet_replica.cc:442] T a3a31565ff894c95b21198a7070b65ef P ec5e0cf1f22e44fba4bd414b63c9cad0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:50:05.800891 23061 tablet_replica.cc:439] T a3a31565ff894c95b21198a7070b65ef P 9f1437b726864fe499094ac03563fe08: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } } }
I20260929 10:50:05.801016 23061 tablet_replica.cc:442] T a3a31565ff894c95b21198a7070b65ef P 9f1437b726864fe499094ac03563fe08: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:50:05.888340 23065 raft_consensus.cc:482] T a3a31565ff894c95b21198a7070b65ef P bb91879b1a8d4c38b9af948109cafe10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:50:05.888451 23065 raft_consensus.cc:504] T a3a31565ff894c95b21198a7070b65ef P bb91879b1a8d4c38b9af948109cafe10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:05.888860 23065 leader_election.cc:288] T a3a31565ff894c95b21198a7070b65ef P bb91879b1a8d4c38b9af948109cafe10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec5e0cf1f22e44fba4bd414b63c9cad0 (127.20.80.193:34995), 9f1437b726864fe499094ac03563fe08 (127.20.80.195:41207)
I20260929 10:50:05.891672 23054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.889233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:33835 (local address 127.20.80.193:34995)
0929 10:50:05.889278 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:05.889280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:05.889289 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:50:05.889331 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:05.889332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:05.889345 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:05.889408 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:05.889555 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.890040 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.890977 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.891328 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.891416 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.891426 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:05.891564 (+ 138us) server_negotiation.cc:299] Negotiation successful
0929 10:50:05.891600 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 10:50:05.891687 23080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.889047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:34995 (local address 127.20.80.194:33835)
0929 10:50:05.889184 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:05.889197 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:05.889255 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:05.889414 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:05.889414 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:05.889417 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:05.889477 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:05.889479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.890124 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.890125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:05.890887 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:05.890889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.891417 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.891418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:05.891493 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.891497 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:50:05.891558 (+ 61us) client_negotiation.cc:240] Negotiation successful
0929 10:50:05.891600 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":60,"threads_started":1}
I20260929 10:50:05.891757 23055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.889150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:47453 (local address 127.20.80.195:41207)
0929 10:50:05.889195 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:05.889198 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:05.889318 (+ 120us) server_negotiation.cc:407] Connection header received
0929 10:50:05.889455 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:05.889456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:05.889466 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:05.889537 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:05.889675 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.890137 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.891037 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.891397 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.891473 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.891481 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:05.891636 (+ 155us) server_negotiation.cc:299] Negotiation successful
0929 10:50:05.891673 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 10:50:05.892033 23079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.889050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:41207 (local address 127.20.80.194:47453)
0929 10:50:05.889259 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:05.889269 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:05.889346 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:05.889540 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:05.889541 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:05.889543 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:05.889605 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:05.889607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.890225 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.890226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:05.890943 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:05.890945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.891503 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.891505 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:05.891565 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.891568 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:50:05.891629 (+ 61us) client_negotiation.cc:240] Negotiation successful
0929 10:50:05.891669 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":59,"threads_started":1}
I20260929 10:50:05.892021 22708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3a31565ff894c95b21198a7070b65ef" candidate_uuid: "bb91879b1a8d4c38b9af948109cafe10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" is_pre_election: true
I20260929 10:50:05.892143 22708 raft_consensus.cc:2454] T a3a31565ff894c95b21198a7070b65ef P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb91879b1a8d4c38b9af948109cafe10 in term 0.
I20260929 10:50:05.892269 22988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3a31565ff894c95b21198a7070b65ef" candidate_uuid: "bb91879b1a8d4c38b9af948109cafe10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1437b726864fe499094ac03563fe08" is_pre_election: true
I20260929 10:50:05.892350 22988 raft_consensus.cc:2454] T a3a31565ff894c95b21198a7070b65ef P 9f1437b726864fe499094ac03563fe08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb91879b1a8d4c38b9af948109cafe10 in term 0.
I20260929 10:50:05.892338 22782 leader_election.cc:302] T a3a31565ff894c95b21198a7070b65ef P bb91879b1a8d4c38b9af948109cafe10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb91879b1a8d4c38b9af948109cafe10, ec5e0cf1f22e44fba4bd414b63c9cad0; no voters:
I20260929 10:50:05.892472 23065 raft_consensus.cc:2781] T a3a31565ff894c95b21198a7070b65ef P bb91879b1a8d4c38b9af948109cafe10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:50:05.892515 23065 raft_consensus.cc:482] T a3a31565ff894c95b21198a7070b65ef P bb91879b1a8d4c38b9af948109cafe10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:50:05.892544 23065 raft_consensus.cc:3037] T a3a31565ff894c95b21198a7070b65ef P bb91879b1a8d4c38b9af948109cafe10 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:05.893278 23065 raft_consensus.cc:504] T a3a31565ff894c95b21198a7070b65ef P bb91879b1a8d4c38b9af948109cafe10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:05.893414 23065 leader_election.cc:288] T a3a31565ff894c95b21198a7070b65ef P bb91879b1a8d4c38b9af948109cafe10 [CANDIDATE]: Term 1 election: Requested vote from peers ec5e0cf1f22e44fba4bd414b63c9cad0 (127.20.80.193:34995), 9f1437b726864fe499094ac03563fe08 (127.20.80.195:41207)
I20260929 10:50:05.893540 22708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3a31565ff894c95b21198a7070b65ef" candidate_uuid: "bb91879b1a8d4c38b9af948109cafe10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0"
I20260929 10:50:05.893553 22988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3a31565ff894c95b21198a7070b65ef" candidate_uuid: "bb91879b1a8d4c38b9af948109cafe10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1437b726864fe499094ac03563fe08"
I20260929 10:50:05.893610 22708 raft_consensus.cc:3037] T a3a31565ff894c95b21198a7070b65ef P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:05.893632 22988 raft_consensus.cc:3037] T a3a31565ff894c95b21198a7070b65ef P 9f1437b726864fe499094ac03563fe08 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:05.894248 22988 raft_consensus.cc:2454] T a3a31565ff894c95b21198a7070b65ef P 9f1437b726864fe499094ac03563fe08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb91879b1a8d4c38b9af948109cafe10 in term 1.
I20260929 10:50:05.894258 22708 raft_consensus.cc:2454] T a3a31565ff894c95b21198a7070b65ef P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb91879b1a8d4c38b9af948109cafe10 in term 1.
I20260929 10:50:05.894430 22779 leader_election.cc:302] T a3a31565ff894c95b21198a7070b65ef P bb91879b1a8d4c38b9af948109cafe10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f1437b726864fe499094ac03563fe08, bb91879b1a8d4c38b9af948109cafe10; no voters:
I20260929 10:50:05.894562 23065 raft_consensus.cc:2781] T a3a31565ff894c95b21198a7070b65ef P bb91879b1a8d4c38b9af948109cafe10 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:05.894680 23065 raft_consensus.cc:686] T a3a31565ff894c95b21198a7070b65ef P bb91879b1a8d4c38b9af948109cafe10 [term 1 LEADER]: Becoming Leader. State: Replica: bb91879b1a8d4c38b9af948109cafe10, State: Running, Role: LEADER
I20260929 10:50:05.894773 23065 consensus_queue.cc:237] T a3a31565ff894c95b21198a7070b65ef P bb91879b1a8d4c38b9af948109cafe10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:05.895198 23081 tablet_replica.cc:439] T a3a31565ff894c95b21198a7070b65ef P bb91879b1a8d4c38b9af948109cafe10: TxnStatusTablet state changed. Reason: New leader bb91879b1a8d4c38b9af948109cafe10. Latest consensus state: current_term: 1 leader_uuid: "bb91879b1a8d4c38b9af948109cafe10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } } }
I20260929 10:50:05.895313 23081 tablet_replica.cc:442] T a3a31565ff894c95b21198a7070b65ef P bb91879b1a8d4c38b9af948109cafe10: This TxnStatusTablet replica's current role is: LEADER
I20260929 10:50:05.895510 23083 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 10:50:05.895623 23083 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 10:50:05.895613 22550 catalog_manager.cc:5495] T a3a31565ff894c95b21198a7070b65ef P bb91879b1a8d4c38b9af948109cafe10 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb91879b1a8d4c38b9af948109cafe10 (127.20.80.194). New cstate: current_term: 1 leader_uuid: "bb91879b1a8d4c38b9af948109cafe10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } health_report { overall_health: UNKNOWN } } }
I20260929 10:50:05.917440 23056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.914937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35998 (local address 127.20.80.194:35241)
0929 10:50:05.914997 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:05.915000 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:05.915009 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:50:05.915087 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:05.915087 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:05.915109 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:05.915173 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:50:05.915311 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.915883 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.916639 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.916833 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.916905 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.917113 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:50:05.917220 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:50:05.917307 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:05.917348 (+ 41us) server_negotiation.cc:299] Negotiation successful
0929 10:50:05.917367 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 10:50:05.917454 23075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.914852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:35241 (local address 127.0.0.1:35998)
0929 10:50:05.914918 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:05.914934 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:05.914999 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:05.915179 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:05.915179 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:05.915181 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:50:05.915244 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:05.915247 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.915964 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.915965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:05.916554 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:05.916556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.916905 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.916906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:05.917011 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.917029 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:50:05.917287 (+ 258us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:50:05.917296 (+ 9us) client_negotiation.cc:769] Sending connection context
0929 10:50:05.917343 (+ 47us) client_negotiation.cc:240] Negotiation successful
0929 10:50:05.917391 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260929 10:50:05.920020 22708 raft_consensus.cc:1261] T a3a31565ff894c95b21198a7070b65ef P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 1 FOLLOWER]: Refusing update from remote peer bb91879b1a8d4c38b9af948109cafe10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:05.920033 22988 raft_consensus.cc:1261] T a3a31565ff894c95b21198a7070b65ef P 9f1437b726864fe499094ac03563fe08 [term 1 FOLLOWER]: Refusing update from remote peer bb91879b1a8d4c38b9af948109cafe10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:05.920359 23081 consensus_queue.cc:1035] T a3a31565ff894c95b21198a7070b65ef P bb91879b1a8d4c38b9af948109cafe10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:50:05.920516 23065 consensus_queue.cc:1035] T a3a31565ff894c95b21198a7070b65ef P bb91879b1a8d4c38b9af948109cafe10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:50:05.921753 23061 tablet_replica.cc:439] T a3a31565ff894c95b21198a7070b65ef P 9f1437b726864fe499094ac03563fe08: TxnStatusTablet state changed. Reason: New leader bb91879b1a8d4c38b9af948109cafe10. Latest consensus state: current_term: 1 leader_uuid: "bb91879b1a8d4c38b9af948109cafe10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } } }
I20260929 10:50:05.921736 23064 tablet_replica.cc:439] T a3a31565ff894c95b21198a7070b65ef P ec5e0cf1f22e44fba4bd414b63c9cad0: TxnStatusTablet state changed. Reason: New leader bb91879b1a8d4c38b9af948109cafe10. Latest consensus state: current_term: 1 leader_uuid: "bb91879b1a8d4c38b9af948109cafe10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } } }
I20260929 10:50:05.921829 23061 tablet_replica.cc:442] T a3a31565ff894c95b21198a7070b65ef P 9f1437b726864fe499094ac03563fe08: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:50:05.921829 23064 tablet_replica.cc:442] T a3a31565ff894c95b21198a7070b65ef P ec5e0cf1f22e44fba4bd414b63c9cad0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:50:05.922458 23065 tablet_replica.cc:439] T a3a31565ff894c95b21198a7070b65ef P bb91879b1a8d4c38b9af948109cafe10: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bb91879b1a8d4c38b9af948109cafe10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } } }
I20260929 10:50:05.922535 23065 tablet_replica.cc:442] T a3a31565ff894c95b21198a7070b65ef P bb91879b1a8d4c38b9af948109cafe10: This TxnStatusTablet replica's current role is: LEADER
I20260929 10:50:05.922706 23086 mvcc.cc:205] Tried to move back new op lower bound from 7334621208245645312 to 7334621208145530880. Current Snapshot: MvccSnapshot[applied={T|T < 7334621208245645312}]
I20260929 10:50:05.923211 23064 tablet_replica.cc:439] T a3a31565ff894c95b21198a7070b65ef P ec5e0cf1f22e44fba4bd414b63c9cad0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bb91879b1a8d4c38b9af948109cafe10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } } }
I20260929 10:50:05.923313 23064 tablet_replica.cc:442] T a3a31565ff894c95b21198a7070b65ef P ec5e0cf1f22e44fba4bd414b63c9cad0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:50:05.923879 23065 tablet_replica.cc:439] T a3a31565ff894c95b21198a7070b65ef P bb91879b1a8d4c38b9af948109cafe10: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bb91879b1a8d4c38b9af948109cafe10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } } }
I20260929 10:50:05.923957 23065 tablet_replica.cc:442] T a3a31565ff894c95b21198a7070b65ef P bb91879b1a8d4c38b9af948109cafe10: This TxnStatusTablet replica's current role is: LEADER
I20260929 10:50:05.924479 23061 tablet_replica.cc:439] T a3a31565ff894c95b21198a7070b65ef P 9f1437b726864fe499094ac03563fe08: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bb91879b1a8d4c38b9af948109cafe10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } } }
I20260929 10:50:05.924584 23061 tablet_replica.cc:442] T a3a31565ff894c95b21198a7070b65ef P 9f1437b726864fe499094ac03563fe08: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:50:05.929294 23054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.926832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35414 (local address 127.20.80.193:34995)
0929 10:50:05.926877 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:05.926879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:05.926901 (+ 22us) server_negotiation.cc:407] Connection header received
0929 10:50:05.926980 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:05.926982 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:05.927001 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:05.927061 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:50:05.927200 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.927742 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.928505 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.928709 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.928797 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.928953 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:50:05.929087 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:50:05.929176 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:05.929196 (+ 20us) server_negotiation.cc:299] Negotiation successful
0929 10:50:05.929218 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 10:50:05.934506 23056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.931903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:52137 (local address 127.20.80.194:35241)
0929 10:50:05.931970 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:05.931972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:05.932104 (+ 132us) server_negotiation.cc:407] Connection header received
0929 10:50:05.932207 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:05.932208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:05.932219 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:05.932274 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:50:05.932410 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.932892 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.933750 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.934015 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.934085 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.934272 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:50:05.934303 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:50:05.934362 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:05.934416 (+ 54us) server_negotiation.cc:299] Negotiation successful
0929 10:50:05.934436 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 10:50:05.934547 23098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.931832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:35241 (local address 127.20.80.193:52137)
0929 10:50:05.932041 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:05.932050 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:05.932132 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:05.932278 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:05.932278 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:05.932279 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:50:05.932334 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:05.932336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.933032 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.933033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:05.933652 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:05.933654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.934085 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.934086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:05.934154 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.934173 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:50:05.934363 (+ 190us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:50:05.934366 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:50:05.934413 (+ 47us) client_negotiation.cc:240] Negotiation successful
0929 10:50:05.934455 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":109,"threads_started":1}
I20260929 10:50:05.941905 22988 raft_consensus.cc:1261] T 5865250ca4e5436f8f38121bb6711b1b P 9f1437b726864fe499094ac03563fe08 [term 1 FOLLOWER]: Refusing update from remote peer ec5e0cf1f22e44fba4bd414b63c9cad0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:05.941965 22848 raft_consensus.cc:1261] T 5865250ca4e5436f8f38121bb6711b1b P bb91879b1a8d4c38b9af948109cafe10 [term 1 FOLLOWER]: Refusing update from remote peer ec5e0cf1f22e44fba4bd414b63c9cad0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:05.942178 23064 consensus_queue.cc:1035] T 5865250ca4e5436f8f38121bb6711b1b P ec5e0cf1f22e44fba4bd414b63c9cad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:50:05.942328 23068 consensus_queue.cc:1035] T 5865250ca4e5436f8f38121bb6711b1b P ec5e0cf1f22e44fba4bd414b63c9cad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:50:05.943630 23087 mvcc.cc:205] Tried to move back new op lower bound from 7334621208336052224 to 7334621207584907264. Current Snapshot: MvccSnapshot[applied={T|T < 7334621208336052224}]
I20260929 10:50:05.951272 23054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.948338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:46527 (local address 127.20.80.193:34995)
0929 10:50:05.948379 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:05.948381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:05.948499 (+ 118us) server_negotiation.cc:407] Connection header received
0929 10:50:05.948579 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:05.948580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:05.948591 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:05.948645 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:50:05.948797 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.949343 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.950460 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.950781 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.950828 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.951038 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:50:05.951070 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:50:05.951115 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:05.951184 (+ 69us) server_negotiation.cc:299] Negotiation successful
0929 10:50:05.951204 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 10:50:05.951330 23102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:05.948299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:34995 (local address 127.20.80.194:46527)
0929 10:50:05.948437 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:05.948451 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:05.948513 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:05.948656 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:05.948657 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:05.948658 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:50:05.948725 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:05.948727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.949447 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.949448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:05.950359 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:05.950361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:05.950885 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:05.950886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:05.950942 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:05.950954 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:50:05.951120 (+ 166us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:50:05.951122 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 10:50:05.951180 (+ 58us) client_negotiation.cc:240] Negotiation successful
0929 10:50:05.951220 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":62,"threads_started":1}
I20260929 10:50:05.959540 22550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52482:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:50:05.959650 22550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52482:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:50:05.960454 22550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0c45c4ef55f34833825183d938c0215c: Sending DeleteTablet for 3 replicas of tablet 5865250ca4e5436f8f38121bb6711b1b
I20260929 10:50:05.960785 22688 tablet_service.cc:1514] Processing DeleteTablet for tablet 5865250ca4e5436f8f38121bb6711b1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:05 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:35410
I20260929 10:50:05.960839 22968 tablet_service.cc:1514] Processing DeleteTablet for tablet 5865250ca4e5436f8f38121bb6711b1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:05 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:58792
I20260929 10:50:05.961081 23108 tablet_replica.cc:330] T 5865250ca4e5436f8f38121bb6711b1b P ec5e0cf1f22e44fba4bd414b63c9cad0: stopping tablet replica
I20260929 10:50:05.961074 22828 tablet_service.cc:1514] Processing DeleteTablet for tablet 5865250ca4e5436f8f38121bb6711b1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:05 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:35996
I20260929 10:50:05.961213 23108 raft_consensus.cc:2229] T 5865250ca4e5436f8f38121bb6711b1b P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:50:05.961302 23108 raft_consensus.cc:2258] T 5865250ca4e5436f8f38121bb6711b1b P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:05.961766 23108 ts_tablet_manager.cc:1905] T 5865250ca4e5436f8f38121bb6711b1b P ec5e0cf1f22e44fba4bd414b63c9cad0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:05.961709 22550 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52482:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:50:05.961848 22550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:50:05.963095 22968 tablet_service.cc:1467] Processing CreateTablet for tablet 5942cb1bfecc430a82d255d547057ea4 (DEFAULT_TABLE table=test-table [id=e189f071c4384dd095eabec2d6347260]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:05.963238 22968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5942cb1bfecc430a82d255d547057ea4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:05.963678 23108 ts_tablet_manager.cc:1918] T 5865250ca4e5436f8f38121bb6711b1b P ec5e0cf1f22e44fba4bd414b63c9cad0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:50:05.963739 23108 log.cc:1196] T 5865250ca4e5436f8f38121bb6711b1b P ec5e0cf1f22e44fba4bd414b63c9cad0: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-0/wal/wals/5865250ca4e5436f8f38121bb6711b1b
I20260929 10:50:05.963997 23108 ts_tablet_manager.cc:1939] T 5865250ca4e5436f8f38121bb6711b1b P ec5e0cf1f22e44fba4bd414b63c9cad0: Deleting consensus metadata
I20260929 10:50:05.964540 22538 catalog_manager.cc:4841] TS ec5e0cf1f22e44fba4bd414b63c9cad0 (127.20.80.193:34995): tablet 5865250ca4e5436f8f38121bb6711b1b (table test-table [id=f479876841b84b55b8e8f0d6367df2df]) successfully deleted
I20260929 10:50:05.964694 23057 tablet_bootstrap.cc:492] T 5942cb1bfecc430a82d255d547057ea4 P 9f1437b726864fe499094ac03563fe08: Bootstrap starting.
I20260929 10:50:05.965088 23057 tablet_bootstrap.cc:654] T 5942cb1bfecc430a82d255d547057ea4 P 9f1437b726864fe499094ac03563fe08: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:05.965395 22688 tablet_service.cc:1467] Processing CreateTablet for tablet 5942cb1bfecc430a82d255d547057ea4 (DEFAULT_TABLE table=test-table [id=e189f071c4384dd095eabec2d6347260]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:05.965493 22688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5942cb1bfecc430a82d255d547057ea4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:05.965482 22828 tablet_service.cc:1467] Processing CreateTablet for tablet 5942cb1bfecc430a82d255d547057ea4 (DEFAULT_TABLE table=test-table [id=e189f071c4384dd095eabec2d6347260]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:05.965597 22828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5942cb1bfecc430a82d255d547057ea4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:05.966653 23058 tablet_bootstrap.cc:492] T 5942cb1bfecc430a82d255d547057ea4 P ec5e0cf1f22e44fba4bd414b63c9cad0: Bootstrap starting.
I20260929 10:50:05.966917 23062 tablet_bootstrap.cc:492] T 5942cb1bfecc430a82d255d547057ea4 P bb91879b1a8d4c38b9af948109cafe10: Bootstrap starting.
I20260929 10:50:05.967047 23058 tablet_bootstrap.cc:654] T 5942cb1bfecc430a82d255d547057ea4 P ec5e0cf1f22e44fba4bd414b63c9cad0: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:05.967298 23062 tablet_bootstrap.cc:654] T 5942cb1bfecc430a82d255d547057ea4 P bb91879b1a8d4c38b9af948109cafe10: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:05.968299 23058 tablet_bootstrap.cc:492] T 5942cb1bfecc430a82d255d547057ea4 P ec5e0cf1f22e44fba4bd414b63c9cad0: No bootstrap required, opened a new log
I20260929 10:50:05.968346 23058 ts_tablet_manager.cc:1397] T 5942cb1bfecc430a82d255d547057ea4 P ec5e0cf1f22e44fba4bd414b63c9cad0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:50:05.968492 23062 tablet_bootstrap.cc:492] T 5942cb1bfecc430a82d255d547057ea4 P bb91879b1a8d4c38b9af948109cafe10: No bootstrap required, opened a new log
I20260929 10:50:05.968477 23058 raft_consensus.cc:348] T 5942cb1bfecc430a82d255d547057ea4 P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:05.968542 23058 raft_consensus.cc:374] T 5942cb1bfecc430a82d255d547057ea4 P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:05.968538 23062 ts_tablet_manager.cc:1397] T 5942cb1bfecc430a82d255d547057ea4 P bb91879b1a8d4c38b9af948109cafe10: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:50:05.968561 23058 raft_consensus.cc:729] T 5942cb1bfecc430a82d255d547057ea4 P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec5e0cf1f22e44fba4bd414b63c9cad0, State: Initialized, Role: FOLLOWER
I20260929 10:50:05.968616 23058 consensus_queue.cc:260] T 5942cb1bfecc430a82d255d547057ea4 P ec5e0cf1f22e44fba4bd414b63c9cad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:05.968704 23058 ts_tablet_manager.cc:1428] T 5942cb1bfecc430a82d255d547057ea4 P ec5e0cf1f22e44fba4bd414b63c9cad0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:05.968686 23062 raft_consensus.cc:348] T 5942cb1bfecc430a82d255d547057ea4 P bb91879b1a8d4c38b9af948109cafe10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:05.968750 23062 raft_consensus.cc:374] T 5942cb1bfecc430a82d255d547057ea4 P bb91879b1a8d4c38b9af948109cafe10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:05.968771 23062 raft_consensus.cc:729] T 5942cb1bfecc430a82d255d547057ea4 P bb91879b1a8d4c38b9af948109cafe10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb91879b1a8d4c38b9af948109cafe10, State: Initialized, Role: FOLLOWER
I20260929 10:50:05.968812 23062 consensus_queue.cc:260] T 5942cb1bfecc430a82d255d547057ea4 P bb91879b1a8d4c38b9af948109cafe10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:05.968899 23062 ts_tablet_manager.cc:1428] T 5942cb1bfecc430a82d255d547057ea4 P bb91879b1a8d4c38b9af948109cafe10: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:05.969552 23109 tablet_replica.cc:330] T 5865250ca4e5436f8f38121bb6711b1b P bb91879b1a8d4c38b9af948109cafe10: stopping tablet replica
I20260929 10:50:05.969704 23109 raft_consensus.cc:2229] T 5865250ca4e5436f8f38121bb6711b1b P bb91879b1a8d4c38b9af948109cafe10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:50:05.969811 23057 tablet_bootstrap.cc:492] T 5942cb1bfecc430a82d255d547057ea4 P 9f1437b726864fe499094ac03563fe08: No bootstrap required, opened a new log
I20260929 10:50:05.969832 23109 raft_consensus.cc:2258] T 5865250ca4e5436f8f38121bb6711b1b P bb91879b1a8d4c38b9af948109cafe10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:05.969866 23057 ts_tablet_manager.cc:1397] T 5942cb1bfecc430a82d255d547057ea4 P 9f1437b726864fe499094ac03563fe08: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260929 10:50:05.970037 23107 tablet_replica.cc:330] T 5865250ca4e5436f8f38121bb6711b1b P 9f1437b726864fe499094ac03563fe08: stopping tablet replica
I20260929 10:50:05.970021 23057 raft_consensus.cc:348] T 5942cb1bfecc430a82d255d547057ea4 P 9f1437b726864fe499094ac03563fe08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:05.970098 23057 raft_consensus.cc:374] T 5942cb1bfecc430a82d255d547057ea4 P 9f1437b726864fe499094ac03563fe08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:05.970121 23057 raft_consensus.cc:729] T 5942cb1bfecc430a82d255d547057ea4 P 9f1437b726864fe499094ac03563fe08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1437b726864fe499094ac03563fe08, State: Initialized, Role: FOLLOWER
I20260929 10:50:05.970163 23057 consensus_queue.cc:260] T 5942cb1bfecc430a82d255d547057ea4 P 9f1437b726864fe499094ac03563fe08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:05.970248 23109 ts_tablet_manager.cc:1905] T 5865250ca4e5436f8f38121bb6711b1b P bb91879b1a8d4c38b9af948109cafe10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:05.970289 23057 ts_tablet_manager.cc:1428] T 5942cb1bfecc430a82d255d547057ea4 P 9f1437b726864fe499094ac03563fe08: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:05.970155 23107 raft_consensus.cc:2229] T 5865250ca4e5436f8f38121bb6711b1b P 9f1437b726864fe499094ac03563fe08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:50:05.970530 23107 raft_consensus.cc:2258] T 5865250ca4e5436f8f38121bb6711b1b P 9f1437b726864fe499094ac03563fe08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:05.970928 23107 ts_tablet_manager.cc:1905] T 5865250ca4e5436f8f38121bb6711b1b P 9f1437b726864fe499094ac03563fe08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:05.971930 23109 ts_tablet_manager.cc:1918] T 5865250ca4e5436f8f38121bb6711b1b P bb91879b1a8d4c38b9af948109cafe10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:50:05.972002 23109 log.cc:1196] T 5865250ca4e5436f8f38121bb6711b1b P bb91879b1a8d4c38b9af948109cafe10: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-1/wal/wals/5865250ca4e5436f8f38121bb6711b1b
I20260929 10:50:05.972240 23109 ts_tablet_manager.cc:1939] T 5865250ca4e5436f8f38121bb6711b1b P bb91879b1a8d4c38b9af948109cafe10: Deleting consensus metadata
I20260929 10:50:05.972600 23107 ts_tablet_manager.cc:1918] T 5865250ca4e5436f8f38121bb6711b1b P 9f1437b726864fe499094ac03563fe08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:50:05.972673 23107 log.cc:1196] T 5865250ca4e5436f8f38121bb6711b1b P 9f1437b726864fe499094ac03563fe08: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-2/wal/wals/5865250ca4e5436f8f38121bb6711b1b
I20260929 10:50:05.972802 22536 catalog_manager.cc:4841] TS bb91879b1a8d4c38b9af948109cafe10 (127.20.80.194:35241): tablet 5865250ca4e5436f8f38121bb6711b1b (table test-table [id=f479876841b84b55b8e8f0d6367df2df]) successfully deleted
I20260929 10:50:05.972970 23107 ts_tablet_manager.cc:1939] T 5865250ca4e5436f8f38121bb6711b1b P 9f1437b726864fe499094ac03563fe08: Deleting consensus metadata
I20260929 10:50:05.973483 22535 catalog_manager.cc:4841] TS 9f1437b726864fe499094ac03563fe08 (127.20.80.195:41207): tablet 5865250ca4e5436f8f38121bb6711b1b (table test-table [id=f479876841b84b55b8e8f0d6367df2df]) successfully deleted
I20260929 10:50:06.038733 23103 raft_consensus.cc:482] T 5942cb1bfecc430a82d255d547057ea4 P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:50:06.038851 23103 raft_consensus.cc:504] T 5942cb1bfecc430a82d255d547057ea4 P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:06.039044 23103 leader_election.cc:288] T 5942cb1bfecc430a82d255d547057ea4 P ec5e0cf1f22e44fba4bd414b63c9cad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb91879b1a8d4c38b9af948109cafe10 (127.20.80.194:35241), 9f1437b726864fe499094ac03563fe08 (127.20.80.195:41207)
I20260929 10:50:06.039266 22848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5942cb1bfecc430a82d255d547057ea4" candidate_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb91879b1a8d4c38b9af948109cafe10" is_pre_election: true
I20260929 10:50:06.039307 22988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5942cb1bfecc430a82d255d547057ea4" candidate_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1437b726864fe499094ac03563fe08" is_pre_election: true
I20260929 10:50:06.039383 22848 raft_consensus.cc:2454] T 5942cb1bfecc430a82d255d547057ea4 P bb91879b1a8d4c38b9af948109cafe10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec5e0cf1f22e44fba4bd414b63c9cad0 in term 0.
I20260929 10:50:06.039397 22988 raft_consensus.cc:2454] T 5942cb1bfecc430a82d255d547057ea4 P 9f1437b726864fe499094ac03563fe08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec5e0cf1f22e44fba4bd414b63c9cad0 in term 0.
I20260929 10:50:06.039544 22640 leader_election.cc:302] T 5942cb1bfecc430a82d255d547057ea4 P ec5e0cf1f22e44fba4bd414b63c9cad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb91879b1a8d4c38b9af948109cafe10, ec5e0cf1f22e44fba4bd414b63c9cad0; no voters:
I20260929 10:50:06.039703 23103 raft_consensus.cc:2781] T 5942cb1bfecc430a82d255d547057ea4 P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:50:06.039772 23103 raft_consensus.cc:482] T 5942cb1bfecc430a82d255d547057ea4 P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:50:06.039812 23103 raft_consensus.cc:3037] T 5942cb1bfecc430a82d255d547057ea4 P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:06.040782 23103 raft_consensus.cc:504] T 5942cb1bfecc430a82d255d547057ea4 P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:06.041113 22848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5942cb1bfecc430a82d255d547057ea4" candidate_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb91879b1a8d4c38b9af948109cafe10"
I20260929 10:50:06.041199 22848 raft_consensus.cc:3037] T 5942cb1bfecc430a82d255d547057ea4 P bb91879b1a8d4c38b9af948109cafe10 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:06.041203 22988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5942cb1bfecc430a82d255d547057ea4" candidate_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1437b726864fe499094ac03563fe08"
I20260929 10:50:06.041271 22988 raft_consensus.cc:3037] T 5942cb1bfecc430a82d255d547057ea4 P 9f1437b726864fe499094ac03563fe08 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:06.042059 22848 raft_consensus.cc:2454] T 5942cb1bfecc430a82d255d547057ea4 P bb91879b1a8d4c38b9af948109cafe10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec5e0cf1f22e44fba4bd414b63c9cad0 in term 1.
I20260929 10:50:06.042075 22988 raft_consensus.cc:2454] T 5942cb1bfecc430a82d255d547057ea4 P 9f1437b726864fe499094ac03563fe08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec5e0cf1f22e44fba4bd414b63c9cad0 in term 1.
I20260929 10:50:06.042389 22640 leader_election.cc:302] T 5942cb1bfecc430a82d255d547057ea4 P ec5e0cf1f22e44fba4bd414b63c9cad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb91879b1a8d4c38b9af948109cafe10, ec5e0cf1f22e44fba4bd414b63c9cad0; no voters:
I20260929 10:50:06.042474 23068 raft_consensus.cc:2781] T 5942cb1bfecc430a82d255d547057ea4 P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:06.042531 23068 raft_consensus.cc:686] T 5942cb1bfecc430a82d255d547057ea4 P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 1 LEADER]: Becoming Leader. State: Replica: ec5e0cf1f22e44fba4bd414b63c9cad0, State: Running, Role: LEADER
I20260929 10:50:06.042600 23068 consensus_queue.cc:237] T 5942cb1bfecc430a82d255d547057ea4 P ec5e0cf1f22e44fba4bd414b63c9cad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:06.042526 23103 leader_election.cc:288] T 5942cb1bfecc430a82d255d547057ea4 P ec5e0cf1f22e44fba4bd414b63c9cad0 [CANDIDATE]: Term 1 election: Requested vote from peers bb91879b1a8d4c38b9af948109cafe10 (127.20.80.194:35241), 9f1437b726864fe499094ac03563fe08 (127.20.80.195:41207)
I20260929 10:50:06.043273 22550 catalog_manager.cc:5495] T 5942cb1bfecc430a82d255d547057ea4 P ec5e0cf1f22e44fba4bd414b63c9cad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec5e0cf1f22e44fba4bd414b63c9cad0 (127.20.80.193). New cstate: current_term: 1 leader_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } health_report { overall_health: UNKNOWN } } }
I20260929 10:50:06.068136 22848 raft_consensus.cc:1261] T 5942cb1bfecc430a82d255d547057ea4 P bb91879b1a8d4c38b9af948109cafe10 [term 1 FOLLOWER]: Refusing update from remote peer ec5e0cf1f22e44fba4bd414b63c9cad0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:06.068259 22988 raft_consensus.cc:1261] T 5942cb1bfecc430a82d255d547057ea4 P 9f1437b726864fe499094ac03563fe08 [term 1 FOLLOWER]: Refusing update from remote peer ec5e0cf1f22e44fba4bd414b63c9cad0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:06.068416 23068 consensus_queue.cc:1035] T 5942cb1bfecc430a82d255d547057ea4 P ec5e0cf1f22e44fba4bd414b63c9cad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:50:06.068521 23103 consensus_queue.cc:1035] T 5942cb1bfecc430a82d255d547057ea4 P ec5e0cf1f22e44fba4bd414b63c9cad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 10:50:06.073632 23087 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260929 10:50:07.592181 22550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52482:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:50:07.592281 22550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52482:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:50:07.593142 22550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0c45c4ef55f34833825183d938c0215c: Sending DeleteTablet for 3 replicas of tablet 5942cb1bfecc430a82d255d547057ea4
I20260929 10:50:07.593403 22688 tablet_service.cc:1514] Processing DeleteTablet for tablet 5942cb1bfecc430a82d255d547057ea4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:07 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:35410
I20260929 10:50:07.593530 22828 tablet_service.cc:1514] Processing DeleteTablet for tablet 5942cb1bfecc430a82d255d547057ea4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:07 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:35996
I20260929 10:50:07.593712 23138 tablet_replica.cc:330] T 5942cb1bfecc430a82d255d547057ea4 P ec5e0cf1f22e44fba4bd414b63c9cad0: stopping tablet replica
I20260929 10:50:07.593693 22968 tablet_service.cc:1514] Processing DeleteTablet for tablet 5942cb1bfecc430a82d255d547057ea4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:07 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:58792
I20260929 10:50:07.593798 23138 raft_consensus.cc:2229] T 5942cb1bfecc430a82d255d547057ea4 P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:50:07.593860 23138 raft_consensus.cc:2258] T 5942cb1bfecc430a82d255d547057ea4 P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:07.593961 23140 tablet_replica.cc:330] T 5942cb1bfecc430a82d255d547057ea4 P 9f1437b726864fe499094ac03563fe08: stopping tablet replica
I20260929 10:50:07.594051 23140 raft_consensus.cc:2229] T 5942cb1bfecc430a82d255d547057ea4 P 9f1437b726864fe499094ac03563fe08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:50:07.594059 23139 tablet_replica.cc:330] T 5942cb1bfecc430a82d255d547057ea4 P bb91879b1a8d4c38b9af948109cafe10: stopping tablet replica
I20260929 10:50:07.594105 23138 ts_tablet_manager.cc:1905] T 5942cb1bfecc430a82d255d547057ea4 P ec5e0cf1f22e44fba4bd414b63c9cad0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:07.594135 23140 raft_consensus.cc:2258] T 5942cb1bfecc430a82d255d547057ea4 P 9f1437b726864fe499094ac03563fe08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:07.594151 23139 raft_consensus.cc:2229] T 5942cb1bfecc430a82d255d547057ea4 P bb91879b1a8d4c38b9af948109cafe10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:50:07.594260 23139 raft_consensus.cc:2258] T 5942cb1bfecc430a82d255d547057ea4 P bb91879b1a8d4c38b9af948109cafe10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:07.594391 23140 ts_tablet_manager.cc:1905] T 5942cb1bfecc430a82d255d547057ea4 P 9f1437b726864fe499094ac03563fe08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:07.596343 23139 ts_tablet_manager.cc:1905] T 5942cb1bfecc430a82d255d547057ea4 P bb91879b1a8d4c38b9af948109cafe10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:07.596377 23138 ts_tablet_manager.cc:1918] T 5942cb1bfecc430a82d255d547057ea4 P ec5e0cf1f22e44fba4bd414b63c9cad0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 10:50:07.596431 23138 log.cc:1196] T 5942cb1bfecc430a82d255d547057ea4 P ec5e0cf1f22e44fba4bd414b63c9cad0: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-0/wal/wals/5942cb1bfecc430a82d255d547057ea4
I20260929 10:50:07.596412 23140 ts_tablet_manager.cc:1918] T 5942cb1bfecc430a82d255d547057ea4 P 9f1437b726864fe499094ac03563fe08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 10:50:07.596549 23140 log.cc:1196] T 5942cb1bfecc430a82d255d547057ea4 P 9f1437b726864fe499094ac03563fe08: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-2/wal/wals/5942cb1bfecc430a82d255d547057ea4
I20260929 10:50:07.596764 23138 ts_tablet_manager.cc:1939] T 5942cb1bfecc430a82d255d547057ea4 P ec5e0cf1f22e44fba4bd414b63c9cad0: Deleting consensus metadata
I20260929 10:50:07.596907 23140 ts_tablet_manager.cc:1939] T 5942cb1bfecc430a82d255d547057ea4 P 9f1437b726864fe499094ac03563fe08: Deleting consensus metadata
I20260929 10:50:07.597206 22538 catalog_manager.cc:4841] TS ec5e0cf1f22e44fba4bd414b63c9cad0 (127.20.80.193:34995): tablet 5942cb1bfecc430a82d255d547057ea4 (table test-table [id=e189f071c4384dd095eabec2d6347260]) successfully deleted
I20260929 10:50:07.597328 22535 catalog_manager.cc:4841] TS 9f1437b726864fe499094ac03563fe08 (127.20.80.195:41207): tablet 5942cb1bfecc430a82d255d547057ea4 (table test-table [id=e189f071c4384dd095eabec2d6347260]) successfully deleted
I20260929 10:50:07.598640 23139 ts_tablet_manager.cc:1918] T 5942cb1bfecc430a82d255d547057ea4 P bb91879b1a8d4c38b9af948109cafe10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 10:50:07.598696 23139 log.cc:1196] T 5942cb1bfecc430a82d255d547057ea4 P bb91879b1a8d4c38b9af948109cafe10: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-1/wal/wals/5942cb1bfecc430a82d255d547057ea4
I20260929 10:50:07.598932 23139 ts_tablet_manager.cc:1939] T 5942cb1bfecc430a82d255d547057ea4 P bb91879b1a8d4c38b9af948109cafe10: Deleting consensus metadata
I20260929 10:50:07.599247 22536 catalog_manager.cc:4841] TS bb91879b1a8d4c38b9af948109cafe10 (127.20.80.194:35241): tablet 5942cb1bfecc430a82d255d547057ea4 (table test-table [id=e189f071c4384dd095eabec2d6347260]) successfully deleted
Sep 29 10:50:07 dist-test-slave-vqch krb5kdc[22506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679007, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 10:50:07 dist-test-slave-vqch krb5kdc[22506](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679007, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:07.623325 23148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:07.610200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52498 (local address 127.20.80.254:43621)
0929 10:50:07.610329 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:07.610331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:07.610340 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:50:07.610398 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:07.610399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:07.610410 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:07.610472 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:07.611112 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:07.611621 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:07.612469 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:07.612662 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:07.612713 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:07.620313 (+ 7600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:07.620315 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:07.620315 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:07.620324 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:07.622090 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:07.622503 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:07.622504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:07.622505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:07.622568 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:07.622759 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:07.622760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:07.622761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:07.622889 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:07.623003 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:07.623105 (+ 102us) server_negotiation.cc:299] Negotiation successful
0929 10:50:07.623202 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":73,"threads_started":1}
I20260929 10:50:07.624899 22550 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52498:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:50:07.625120 22550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:50:07.626667 22688 tablet_service.cc:1467] Processing CreateTablet for tablet 7bbb951062b349d88072ebc2c01ba696 (DEFAULT_TABLE table=test-table [id=bf30d6b6228e4364b569c0b22e014cf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:07.626724 22828 tablet_service.cc:1467] Processing CreateTablet for tablet 7bbb951062b349d88072ebc2c01ba696 (DEFAULT_TABLE table=test-table [id=bf30d6b6228e4364b569c0b22e014cf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:07.626832 22688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bbb951062b349d88072ebc2c01ba696. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:07.626888 22828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bbb951062b349d88072ebc2c01ba696. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:07.628048 23149 tablet_bootstrap.cc:492] T 7bbb951062b349d88072ebc2c01ba696 P ec5e0cf1f22e44fba4bd414b63c9cad0: Bootstrap starting.
I20260929 10:50:07.628118 22968 tablet_service.cc:1467] Processing CreateTablet for tablet 7bbb951062b349d88072ebc2c01ba696 (DEFAULT_TABLE table=test-table [id=bf30d6b6228e4364b569c0b22e014cf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:07.628296 22968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bbb951062b349d88072ebc2c01ba696. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:07.628489 23149 tablet_bootstrap.cc:654] T 7bbb951062b349d88072ebc2c01ba696 P ec5e0cf1f22e44fba4bd414b63c9cad0: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:07.629106 23149 tablet_bootstrap.cc:492] T 7bbb951062b349d88072ebc2c01ba696 P ec5e0cf1f22e44fba4bd414b63c9cad0: No bootstrap required, opened a new log
I20260929 10:50:07.629158 23149 ts_tablet_manager.cc:1397] T 7bbb951062b349d88072ebc2c01ba696 P ec5e0cf1f22e44fba4bd414b63c9cad0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:07.629320 23149 raft_consensus.cc:348] T 7bbb951062b349d88072ebc2c01ba696 P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:07.629388 23149 raft_consensus.cc:374] T 7bbb951062b349d88072ebc2c01ba696 P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:07.629403 23149 raft_consensus.cc:729] T 7bbb951062b349d88072ebc2c01ba696 P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec5e0cf1f22e44fba4bd414b63c9cad0, State: Initialized, Role: FOLLOWER
I20260929 10:50:07.629472 23149 consensus_queue.cc:260] T 7bbb951062b349d88072ebc2c01ba696 P ec5e0cf1f22e44fba4bd414b63c9cad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:07.629581 23149 ts_tablet_manager.cc:1428] T 7bbb951062b349d88072ebc2c01ba696 P ec5e0cf1f22e44fba4bd414b63c9cad0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:07.630013 23150 tablet_bootstrap.cc:492] T 7bbb951062b349d88072ebc2c01ba696 P bb91879b1a8d4c38b9af948109cafe10: Bootstrap starting.
I20260929 10:50:07.630188 23152 tablet_bootstrap.cc:492] T 7bbb951062b349d88072ebc2c01ba696 P 9f1437b726864fe499094ac03563fe08: Bootstrap starting.
I20260929 10:50:07.630487 23150 tablet_bootstrap.cc:654] T 7bbb951062b349d88072ebc2c01ba696 P bb91879b1a8d4c38b9af948109cafe10: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:07.630584 23152 tablet_bootstrap.cc:654] T 7bbb951062b349d88072ebc2c01ba696 P 9f1437b726864fe499094ac03563fe08: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:07.631089 23150 tablet_bootstrap.cc:492] T 7bbb951062b349d88072ebc2c01ba696 P bb91879b1a8d4c38b9af948109cafe10: No bootstrap required, opened a new log
I20260929 10:50:07.631153 23150 ts_tablet_manager.cc:1397] T 7bbb951062b349d88072ebc2c01ba696 P bb91879b1a8d4c38b9af948109cafe10: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:07.631240 23152 tablet_bootstrap.cc:492] T 7bbb951062b349d88072ebc2c01ba696 P 9f1437b726864fe499094ac03563fe08: No bootstrap required, opened a new log
I20260929 10:50:07.631299 23152 ts_tablet_manager.cc:1397] T 7bbb951062b349d88072ebc2c01ba696 P 9f1437b726864fe499094ac03563fe08: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:07.631278 23150 raft_consensus.cc:348] T 7bbb951062b349d88072ebc2c01ba696 P bb91879b1a8d4c38b9af948109cafe10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:07.631379 23150 raft_consensus.cc:374] T 7bbb951062b349d88072ebc2c01ba696 P bb91879b1a8d4c38b9af948109cafe10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:07.631412 23150 raft_consensus.cc:729] T 7bbb951062b349d88072ebc2c01ba696 P bb91879b1a8d4c38b9af948109cafe10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb91879b1a8d4c38b9af948109cafe10, State: Initialized, Role: FOLLOWER
I20260929 10:50:07.631462 23150 consensus_queue.cc:260] T 7bbb951062b349d88072ebc2c01ba696 P bb91879b1a8d4c38b9af948109cafe10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:07.631472 23152 raft_consensus.cc:348] T 7bbb951062b349d88072ebc2c01ba696 P 9f1437b726864fe499094ac03563fe08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:07.631548 23152 raft_consensus.cc:374] T 7bbb951062b349d88072ebc2c01ba696 P 9f1437b726864fe499094ac03563fe08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:07.631564 23150 ts_tablet_manager.cc:1428] T 7bbb951062b349d88072ebc2c01ba696 P bb91879b1a8d4c38b9af948109cafe10: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:07.631583 23152 raft_consensus.cc:729] T 7bbb951062b349d88072ebc2c01ba696 P 9f1437b726864fe499094ac03563fe08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1437b726864fe499094ac03563fe08, State: Initialized, Role: FOLLOWER
I20260929 10:50:07.631632 23152 consensus_queue.cc:260] T 7bbb951062b349d88072ebc2c01ba696 P 9f1437b726864fe499094ac03563fe08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:07.631894 23152 ts_tablet_manager.cc:1428] T 7bbb951062b349d88072ebc2c01ba696 P 9f1437b726864fe499094ac03563fe08: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:07.650027 23155 raft_consensus.cc:482] T 7bbb951062b349d88072ebc2c01ba696 P 9f1437b726864fe499094ac03563fe08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:50:07.650125 23155 raft_consensus.cc:504] T 7bbb951062b349d88072ebc2c01ba696 P 9f1437b726864fe499094ac03563fe08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:07.650455 23155 leader_election.cc:288] T 7bbb951062b349d88072ebc2c01ba696 P 9f1437b726864fe499094ac03563fe08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec5e0cf1f22e44fba4bd414b63c9cad0 (127.20.80.193:34995), bb91879b1a8d4c38b9af948109cafe10 (127.20.80.194:35241)
I20260929 10:50:07.653584 23159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:07.650758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:56589 (local address 127.20.80.193:34995)
0929 10:50:07.650908 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:07.650910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:07.650919 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:50:07.651009 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:07.651010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:07.651020 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:07.651105 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:07.651250 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:07.651726 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:07.652792 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:07.653224 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:07.653308 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:07.653318 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:07.653459 (+ 141us) server_negotiation.cc:299] Negotiation successful
0929 10:50:07.653502 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":79,"threads_started":1}
I20260929 10:50:07.653600 23158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:07.650673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:34229 (local address 127.20.80.194:35241)
0929 10:50:07.650839 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:07.650842 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:07.650853 (+ 11us) server_negotiation.cc:407] Connection header received
0929 10:50:07.650940 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:07.650940 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:07.650953 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:07.651030 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:07.651195 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:07.651677 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:07.652779 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:07.653217 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:07.653319 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:07.653328 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:07.653480 (+ 152us) server_negotiation.cc:299] Negotiation successful
0929 10:50:07.653526 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":106,"threads_started":1}
I20260929 10:50:07.653635 23156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:07.650612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:35241 (local address 127.20.80.195:34229)
0929 10:50:07.650805 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:07.650817 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:07.650882 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:07.651031 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:07.651032 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:07.651040 (+ 8us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:07.651113 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:07.651115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:07.651757 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:07.651758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:07.652691 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:07.652693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:07.653318 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:07.653319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:07.653399 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:07.653402 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:50:07.653472 (+ 70us) client_negotiation.cc:240] Negotiation successful
0929 10:50:07.653515 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":1,"thread_start_us":113,"threads_started":1}
I20260929 10:50:07.653826 23157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:07.650612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:34995 (local address 127.20.80.195:56589)
0929 10:50:07.650858 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:07.650870 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:07.650932 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:07.651108 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:07.651109 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:07.651111 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:07.651182 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:07.651184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:07.651806 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:07.651807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:07.652688 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:07.652691 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:07.653307 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:07.653308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:07.653378 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:07.653381 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:50:07.653451 (+ 70us) client_negotiation.cc:240] Negotiation successful
0929 10:50:07.653494 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":3,"thread_start_us":110,"threads_started":1}
I20260929 10:50:07.653816 22848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bbb951062b349d88072ebc2c01ba696" candidate_uuid: "9f1437b726864fe499094ac03563fe08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb91879b1a8d4c38b9af948109cafe10" is_pre_election: true
I20260929 10:50:07.653894 22848 raft_consensus.cc:2454] T 7bbb951062b349d88072ebc2c01ba696 P bb91879b1a8d4c38b9af948109cafe10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1437b726864fe499094ac03563fe08 in term 0.
I20260929 10:50:07.654090 22920 leader_election.cc:302] T 7bbb951062b349d88072ebc2c01ba696 P 9f1437b726864fe499094ac03563fe08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f1437b726864fe499094ac03563fe08, bb91879b1a8d4c38b9af948109cafe10; no voters:
I20260929 10:50:07.654119 22708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bbb951062b349d88072ebc2c01ba696" candidate_uuid: "9f1437b726864fe499094ac03563fe08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" is_pre_election: true
I20260929 10:50:07.654196 22708 raft_consensus.cc:2454] T 7bbb951062b349d88072ebc2c01ba696 P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1437b726864fe499094ac03563fe08 in term 0.
I20260929 10:50:07.654223 23155 raft_consensus.cc:2781] T 7bbb951062b349d88072ebc2c01ba696 P 9f1437b726864fe499094ac03563fe08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:50:07.654263 23155 raft_consensus.cc:482] T 7bbb951062b349d88072ebc2c01ba696 P 9f1437b726864fe499094ac03563fe08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:50:07.654294 23155 raft_consensus.cc:3037] T 7bbb951062b349d88072ebc2c01ba696 P 9f1437b726864fe499094ac03563fe08 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:07.655094 23155 raft_consensus.cc:504] T 7bbb951062b349d88072ebc2c01ba696 P 9f1437b726864fe499094ac03563fe08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:07.655242 23155 leader_election.cc:288] T 7bbb951062b349d88072ebc2c01ba696 P 9f1437b726864fe499094ac03563fe08 [CANDIDATE]: Term 1 election: Requested vote from peers ec5e0cf1f22e44fba4bd414b63c9cad0 (127.20.80.193:34995), bb91879b1a8d4c38b9af948109cafe10 (127.20.80.194:35241)
I20260929 10:50:07.655396 22708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bbb951062b349d88072ebc2c01ba696" candidate_uuid: "9f1437b726864fe499094ac03563fe08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0"
I20260929 10:50:07.655411 22848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bbb951062b349d88072ebc2c01ba696" candidate_uuid: "9f1437b726864fe499094ac03563fe08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb91879b1a8d4c38b9af948109cafe10"
I20260929 10:50:07.655475 22708 raft_consensus.cc:3037] T 7bbb951062b349d88072ebc2c01ba696 P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:07.655488 22848 raft_consensus.cc:3037] T 7bbb951062b349d88072ebc2c01ba696 P bb91879b1a8d4c38b9af948109cafe10 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:07.656216 22708 raft_consensus.cc:2454] T 7bbb951062b349d88072ebc2c01ba696 P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1437b726864fe499094ac03563fe08 in term 1.
I20260929 10:50:07.656206 22848 raft_consensus.cc:2454] T 7bbb951062b349d88072ebc2c01ba696 P bb91879b1a8d4c38b9af948109cafe10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1437b726864fe499094ac03563fe08 in term 1.
I20260929 10:50:07.656365 22920 leader_election.cc:302] T 7bbb951062b349d88072ebc2c01ba696 P 9f1437b726864fe499094ac03563fe08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f1437b726864fe499094ac03563fe08, bb91879b1a8d4c38b9af948109cafe10; no voters:
I20260929 10:50:07.656462 23155 raft_consensus.cc:2781] T 7bbb951062b349d88072ebc2c01ba696 P 9f1437b726864fe499094ac03563fe08 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:07.656589 23155 raft_consensus.cc:686] T 7bbb951062b349d88072ebc2c01ba696 P 9f1437b726864fe499094ac03563fe08 [term 1 LEADER]: Becoming Leader. State: Replica: 9f1437b726864fe499094ac03563fe08, State: Running, Role: LEADER
I20260929 10:50:07.656658 23155 consensus_queue.cc:237] T 7bbb951062b349d88072ebc2c01ba696 P 9f1437b726864fe499094ac03563fe08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:07.657330 22551 catalog_manager.cc:5495] T 7bbb951062b349d88072ebc2c01ba696 P 9f1437b726864fe499094ac03563fe08 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f1437b726864fe499094ac03563fe08 (127.20.80.195). New cstate: current_term: 1 leader_uuid: "9f1437b726864fe499094ac03563fe08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } health_report { overall_health: HEALTHY } } }
I20260929 10:50:07.668232 23163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:07.665870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58798 (local address 127.20.80.195:41207)
0929 10:50:07.666051 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:07.666053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:07.666061 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:50:07.666092 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:07.666092 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:07.666114 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:07.666195 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:50:07.666305 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:07.666820 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:07.667534 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:07.667748 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:07.667799 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:07.667926 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:50:07.668024 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:50:07.668074 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:07.668104 (+ 30us) server_negotiation.cc:299] Negotiation successful
0929 10:50:07.668118 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":92,"threads_started":1}
I20260929 10:50:07.673249 23158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:07.670656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:50633 (local address 127.20.80.194:35241)
0929 10:50:07.670752 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:07.670754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:07.670889 (+ 135us) server_negotiation.cc:407] Connection header received
0929 10:50:07.670977 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:07.670978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:07.670989 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:07.671059 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:50:07.671206 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:07.671694 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:07.672508 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:07.672698 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:07.672766 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:07.672954 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:50:07.673030 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:50:07.673091 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:07.673155 (+ 64us) server_negotiation.cc:299] Negotiation successful
0929 10:50:07.673177 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 10:50:07.673275 23166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:07.670575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:35241 (local address 127.20.80.195:50633)
0929 10:50:07.670796 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:07.670807 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:07.670907 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:07.671061 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:07.671062 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:07.671063 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:50:07.671135 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:07.671136 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:07.671774 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:07.671775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:07.672420 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:07.672422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:07.672788 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:07.672789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:07.672853 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:07.672866 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:50:07.673101 (+ 235us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:50:07.673104 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:50:07.673152 (+ 48us) client_negotiation.cc:240] Negotiation successful
0929 10:50:07.673193 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":90,"threads_started":1}
I20260929 10:50:07.675323 22708 raft_consensus.cc:1261] T 7bbb951062b349d88072ebc2c01ba696 P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 1 FOLLOWER]: Refusing update from remote peer 9f1437b726864fe499094ac03563fe08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:07.675344 22848 raft_consensus.cc:1261] T 7bbb951062b349d88072ebc2c01ba696 P bb91879b1a8d4c38b9af948109cafe10 [term 1 FOLLOWER]: Refusing update from remote peer 9f1437b726864fe499094ac03563fe08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:07.675587 23155 consensus_queue.cc:1035] T 7bbb951062b349d88072ebc2c01ba696 P 9f1437b726864fe499094ac03563fe08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:50:07.675724 23160 consensus_queue.cc:1035] T 7bbb951062b349d88072ebc2c01ba696 P 9f1437b726864fe499094ac03563fe08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:50:07.676903 23120 mvcc.cc:205] Tried to move back new op lower bound from 7334621215436472320 to 7334621215362179072. Current Snapshot: MvccSnapshot[applied={T|T < 7334621215436472320}]
I20260929 10:50:07.682858 23163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:07.680516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:42751 (local address 127.20.80.195:41207)
0929 10:50:07.680566 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:07.680568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:07.680666 (+ 98us) server_negotiation.cc:407] Connection header received
0929 10:50:07.680727 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:07.680727 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:07.680739 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:07.680791 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:50:07.680893 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:07.681407 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:07.682161 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:07.682468 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:07.682528 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:07.682675 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:50:07.682710 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:50:07.682761 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:07.682793 (+ 32us) server_negotiation.cc:299] Negotiation successful
0929 10:50:07.682812 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 10:50:07.682889 23171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:07.680471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:41207 (local address 127.20.80.194:42751)
0929 10:50:07.680613 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:07.680623 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:07.680678 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:07.680794 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:07.680794 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:07.680796 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:50:07.680847 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:07.680848 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:07.681482 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:07.681483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:07.682071 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:07.682072 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:07.682543 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:07.682544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:07.682601 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:07.682606 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:50:07.682760 (+ 154us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:50:07.682761 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 10:50:07.682792 (+ 31us) client_negotiation.cc:240] Negotiation successful
0929 10:50:07.682827 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":85,"threads_started":1}
I20260929 10:50:07.689303 22549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:50:07.689420 22549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:50:07.690040 22549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0c45c4ef55f34833825183d938c0215c: Sending DeleteTablet for 3 replicas of tablet 7bbb951062b349d88072ebc2c01ba696
I20260929 10:50:07.690295 22828 tablet_service.cc:1514] Processing DeleteTablet for tablet 7bbb951062b349d88072ebc2c01ba696 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:07 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:35996
I20260929 10:50:07.690313 22688 tablet_service.cc:1514] Processing DeleteTablet for tablet 7bbb951062b349d88072ebc2c01ba696 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:07 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:35410
I20260929 10:50:07.690398 23139 tablet_replica.cc:330] T 7bbb951062b349d88072ebc2c01ba696 P bb91879b1a8d4c38b9af948109cafe10: stopping tablet replica
I20260929 10:50:07.690407 23138 tablet_replica.cc:330] T 7bbb951062b349d88072ebc2c01ba696 P ec5e0cf1f22e44fba4bd414b63c9cad0: stopping tablet replica
I20260929 10:50:07.690457 23138 raft_consensus.cc:2229] T 7bbb951062b349d88072ebc2c01ba696 P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:50:07.690457 23139 raft_consensus.cc:2229] T 7bbb951062b349d88072ebc2c01ba696 P bb91879b1a8d4c38b9af948109cafe10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:50:07.690488 23138 raft_consensus.cc:2258] T 7bbb951062b349d88072ebc2c01ba696 P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:07.690493 23139 raft_consensus.cc:2258] T 7bbb951062b349d88072ebc2c01ba696 P bb91879b1a8d4c38b9af948109cafe10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:07.690567 22968 tablet_service.cc:1514] Processing DeleteTablet for tablet 7bbb951062b349d88072ebc2c01ba696 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:07 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:58792
I20260929 10:50:07.690645 23140 tablet_replica.cc:330] T 7bbb951062b349d88072ebc2c01ba696 P 9f1437b726864fe499094ac03563fe08: stopping tablet replica
I20260929 10:50:07.690692 23140 raft_consensus.cc:2229] T 7bbb951062b349d88072ebc2c01ba696 P 9f1437b726864fe499094ac03563fe08 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:50:07.690783 23140 raft_consensus.cc:2258] T 7bbb951062b349d88072ebc2c01ba696 P 9f1437b726864fe499094ac03563fe08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:07.690860 23139 ts_tablet_manager.cc:1905] T 7bbb951062b349d88072ebc2c01ba696 P bb91879b1a8d4c38b9af948109cafe10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:07.691058 23140 ts_tablet_manager.cc:1905] T 7bbb951062b349d88072ebc2c01ba696 P 9f1437b726864fe499094ac03563fe08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:07.692658 23140 ts_tablet_manager.cc:1918] T 7bbb951062b349d88072ebc2c01ba696 P 9f1437b726864fe499094ac03563fe08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:50:07.692663 23139 ts_tablet_manager.cc:1918] T 7bbb951062b349d88072ebc2c01ba696 P bb91879b1a8d4c38b9af948109cafe10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:50:07.692723 23139 log.cc:1196] T 7bbb951062b349d88072ebc2c01ba696 P bb91879b1a8d4c38b9af948109cafe10: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-1/wal/wals/7bbb951062b349d88072ebc2c01ba696
I20260929 10:50:07.692735 23140 log.cc:1196] T 7bbb951062b349d88072ebc2c01ba696 P 9f1437b726864fe499094ac03563fe08: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-2/wal/wals/7bbb951062b349d88072ebc2c01ba696
I20260929 10:50:07.692806 22549 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52498:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 10:50:07.692965 23139 ts_tablet_manager.cc:1939] T 7bbb951062b349d88072ebc2c01ba696 P bb91879b1a8d4c38b9af948109cafe10: Deleting consensus metadata
W20260929 10:50:07.692999 22549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:50:07.693024 23140 ts_tablet_manager.cc:1939] T 7bbb951062b349d88072ebc2c01ba696 P 9f1437b726864fe499094ac03563fe08: Deleting consensus metadata
I20260929 10:50:07.693328 22536 catalog_manager.cc:4841] TS bb91879b1a8d4c38b9af948109cafe10 (127.20.80.194:35241): tablet 7bbb951062b349d88072ebc2c01ba696 (table test-table [id=bf30d6b6228e4364b569c0b22e014cf5]) successfully deleted
I20260929 10:50:07.693636 22535 catalog_manager.cc:4841] TS 9f1437b726864fe499094ac03563fe08 (127.20.80.195:41207): tablet 7bbb951062b349d88072ebc2c01ba696 (table test-table [id=bf30d6b6228e4364b569c0b22e014cf5]) successfully deleted
I20260929 10:50:07.693900 23138 ts_tablet_manager.cc:1905] T 7bbb951062b349d88072ebc2c01ba696 P ec5e0cf1f22e44fba4bd414b63c9cad0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:07.694514 22688 tablet_service.cc:1467] Processing CreateTablet for tablet 2ddf84e9e05448098a356fddfa659b96 (DEFAULT_TABLE table=test-table [id=c329eb12e01644cb93bb01a8d2b7c0b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:07.694533 22828 tablet_service.cc:1467] Processing CreateTablet for tablet 2ddf84e9e05448098a356fddfa659b96 (DEFAULT_TABLE table=test-table [id=c329eb12e01644cb93bb01a8d2b7c0b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:07.694621 22688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ddf84e9e05448098a356fddfa659b96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:07.694633 22828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ddf84e9e05448098a356fddfa659b96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:07.695752 23138 ts_tablet_manager.cc:1918] T 7bbb951062b349d88072ebc2c01ba696 P ec5e0cf1f22e44fba4bd414b63c9cad0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:50:07.695803 23138 log.cc:1196] T 7bbb951062b349d88072ebc2c01ba696 P ec5e0cf1f22e44fba4bd414b63c9cad0: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-0/wal/wals/7bbb951062b349d88072ebc2c01ba696
I20260929 10:50:07.695930 23150 tablet_bootstrap.cc:492] T 2ddf84e9e05448098a356fddfa659b96 P bb91879b1a8d4c38b9af948109cafe10: Bootstrap starting.
I20260929 10:50:07.695964 23149 tablet_bootstrap.cc:492] T 2ddf84e9e05448098a356fddfa659b96 P ec5e0cf1f22e44fba4bd414b63c9cad0: Bootstrap starting.
I20260929 10:50:07.696029 23138 ts_tablet_manager.cc:1939] T 7bbb951062b349d88072ebc2c01ba696 P ec5e0cf1f22e44fba4bd414b63c9cad0: Deleting consensus metadata
I20260929 10:50:07.696264 23149 tablet_bootstrap.cc:654] T 2ddf84e9e05448098a356fddfa659b96 P ec5e0cf1f22e44fba4bd414b63c9cad0: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:07.696264 23150 tablet_bootstrap.cc:654] T 2ddf84e9e05448098a356fddfa659b96 P bb91879b1a8d4c38b9af948109cafe10: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:07.696501 22968 tablet_service.cc:1467] Processing CreateTablet for tablet 2ddf84e9e05448098a356fddfa659b96 (DEFAULT_TABLE table=test-table [id=c329eb12e01644cb93bb01a8d2b7c0b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:07.696602 22968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ddf84e9e05448098a356fddfa659b96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:07.696661 22538 catalog_manager.cc:4841] TS ec5e0cf1f22e44fba4bd414b63c9cad0 (127.20.80.193:34995): tablet 7bbb951062b349d88072ebc2c01ba696 (table test-table [id=bf30d6b6228e4364b569c0b22e014cf5]) successfully deleted
I20260929 10:50:07.696998 23149 tablet_bootstrap.cc:492] T 2ddf84e9e05448098a356fddfa659b96 P ec5e0cf1f22e44fba4bd414b63c9cad0: No bootstrap required, opened a new log
I20260929 10:50:07.697026 23150 tablet_bootstrap.cc:492] T 2ddf84e9e05448098a356fddfa659b96 P bb91879b1a8d4c38b9af948109cafe10: No bootstrap required, opened a new log
I20260929 10:50:07.697039 23149 ts_tablet_manager.cc:1397] T 2ddf84e9e05448098a356fddfa659b96 P ec5e0cf1f22e44fba4bd414b63c9cad0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:07.697067 23150 ts_tablet_manager.cc:1397] T 2ddf84e9e05448098a356fddfa659b96 P bb91879b1a8d4c38b9af948109cafe10: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:07.697144 23149 raft_consensus.cc:348] T 2ddf84e9e05448098a356fddfa659b96 P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:07.697218 23149 raft_consensus.cc:374] T 2ddf84e9e05448098a356fddfa659b96 P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:07.697194 23150 raft_consensus.cc:348] T 2ddf84e9e05448098a356fddfa659b96 P bb91879b1a8d4c38b9af948109cafe10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:07.697244 23149 raft_consensus.cc:729] T 2ddf84e9e05448098a356fddfa659b96 P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec5e0cf1f22e44fba4bd414b63c9cad0, State: Initialized, Role: FOLLOWER
I20260929 10:50:07.697260 23150 raft_consensus.cc:374] T 2ddf84e9e05448098a356fddfa659b96 P bb91879b1a8d4c38b9af948109cafe10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:07.697281 23150 raft_consensus.cc:729] T 2ddf84e9e05448098a356fddfa659b96 P bb91879b1a8d4c38b9af948109cafe10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb91879b1a8d4c38b9af948109cafe10, State: Initialized, Role: FOLLOWER
I20260929 10:50:07.697301 23149 consensus_queue.cc:260] T 2ddf84e9e05448098a356fddfa659b96 P ec5e0cf1f22e44fba4bd414b63c9cad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:07.697328 23150 consensus_queue.cc:260] T 2ddf84e9e05448098a356fddfa659b96 P bb91879b1a8d4c38b9af948109cafe10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:07.697412 23149 ts_tablet_manager.cc:1428] T 2ddf84e9e05448098a356fddfa659b96 P ec5e0cf1f22e44fba4bd414b63c9cad0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:07.697423 23150 ts_tablet_manager.cc:1428] T 2ddf84e9e05448098a356fddfa659b96 P bb91879b1a8d4c38b9af948109cafe10: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:07.697870 23152 tablet_bootstrap.cc:492] T 2ddf84e9e05448098a356fddfa659b96 P 9f1437b726864fe499094ac03563fe08: Bootstrap starting.
I20260929 10:50:07.698262 23152 tablet_bootstrap.cc:654] T 2ddf84e9e05448098a356fddfa659b96 P 9f1437b726864fe499094ac03563fe08: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:07.698873 23152 tablet_bootstrap.cc:492] T 2ddf84e9e05448098a356fddfa659b96 P 9f1437b726864fe499094ac03563fe08: No bootstrap required, opened a new log
I20260929 10:50:07.698946 23152 ts_tablet_manager.cc:1397] T 2ddf84e9e05448098a356fddfa659b96 P 9f1437b726864fe499094ac03563fe08: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:07.699112 23152 raft_consensus.cc:348] T 2ddf84e9e05448098a356fddfa659b96 P 9f1437b726864fe499094ac03563fe08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:07.699164 23152 raft_consensus.cc:374] T 2ddf84e9e05448098a356fddfa659b96 P 9f1437b726864fe499094ac03563fe08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:07.699200 23152 raft_consensus.cc:729] T 2ddf84e9e05448098a356fddfa659b96 P 9f1437b726864fe499094ac03563fe08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1437b726864fe499094ac03563fe08, State: Initialized, Role: FOLLOWER
I20260929 10:50:07.699259 23152 consensus_queue.cc:260] T 2ddf84e9e05448098a356fddfa659b96 P 9f1437b726864fe499094ac03563fe08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:07.699365 23152 ts_tablet_manager.cc:1428] T 2ddf84e9e05448098a356fddfa659b96 P 9f1437b726864fe499094ac03563fe08: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:07.802979 23133 raft_consensus.cc:482] T 2ddf84e9e05448098a356fddfa659b96 P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:50:07.803099 23133 raft_consensus.cc:504] T 2ddf84e9e05448098a356fddfa659b96 P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:07.803330 23133 leader_election.cc:288] T 2ddf84e9e05448098a356fddfa659b96 P ec5e0cf1f22e44fba4bd414b63c9cad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb91879b1a8d4c38b9af948109cafe10 (127.20.80.194:35241), 9f1437b726864fe499094ac03563fe08 (127.20.80.195:41207)
I20260929 10:50:07.803536 22848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ddf84e9e05448098a356fddfa659b96" candidate_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb91879b1a8d4c38b9af948109cafe10" is_pre_election: true
I20260929 10:50:07.803535 22988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ddf84e9e05448098a356fddfa659b96" candidate_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1437b726864fe499094ac03563fe08" is_pre_election: true
I20260929 10:50:07.803642 22988 raft_consensus.cc:2454] T 2ddf84e9e05448098a356fddfa659b96 P 9f1437b726864fe499094ac03563fe08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec5e0cf1f22e44fba4bd414b63c9cad0 in term 0.
I20260929 10:50:07.803644 22848 raft_consensus.cc:2454] T 2ddf84e9e05448098a356fddfa659b96 P bb91879b1a8d4c38b9af948109cafe10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec5e0cf1f22e44fba4bd414b63c9cad0 in term 0.
I20260929 10:50:07.803802 22639 leader_election.cc:302] T 2ddf84e9e05448098a356fddfa659b96 P ec5e0cf1f22e44fba4bd414b63c9cad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f1437b726864fe499094ac03563fe08, ec5e0cf1f22e44fba4bd414b63c9cad0; no voters:
I20260929 10:50:07.803922 23133 raft_consensus.cc:2781] T 2ddf84e9e05448098a356fddfa659b96 P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:50:07.803984 23133 raft_consensus.cc:482] T 2ddf84e9e05448098a356fddfa659b96 P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:50:07.804011 23133 raft_consensus.cc:3037] T 2ddf84e9e05448098a356fddfa659b96 P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:07.804759 23133 raft_consensus.cc:504] T 2ddf84e9e05448098a356fddfa659b96 P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:07.804908 23133 leader_election.cc:288] T 2ddf84e9e05448098a356fddfa659b96 P ec5e0cf1f22e44fba4bd414b63c9cad0 [CANDIDATE]: Term 1 election: Requested vote from peers bb91879b1a8d4c38b9af948109cafe10 (127.20.80.194:35241), 9f1437b726864fe499094ac03563fe08 (127.20.80.195:41207)
I20260929 10:50:07.805056 22848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ddf84e9e05448098a356fddfa659b96" candidate_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb91879b1a8d4c38b9af948109cafe10"
I20260929 10:50:07.805126 22848 raft_consensus.cc:3037] T 2ddf84e9e05448098a356fddfa659b96 P bb91879b1a8d4c38b9af948109cafe10 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:07.805116 22988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ddf84e9e05448098a356fddfa659b96" candidate_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1437b726864fe499094ac03563fe08"
I20260929 10:50:07.805193 22988 raft_consensus.cc:3037] T 2ddf84e9e05448098a356fddfa659b96 P 9f1437b726864fe499094ac03563fe08 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:07.805745 22848 raft_consensus.cc:2454] T 2ddf84e9e05448098a356fddfa659b96 P bb91879b1a8d4c38b9af948109cafe10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec5e0cf1f22e44fba4bd414b63c9cad0 in term 1.
I20260929 10:50:07.805907 22640 leader_election.cc:302] T 2ddf84e9e05448098a356fddfa659b96 P ec5e0cf1f22e44fba4bd414b63c9cad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb91879b1a8d4c38b9af948109cafe10, ec5e0cf1f22e44fba4bd414b63c9cad0; no voters:
I20260929 10:50:07.805951 22988 raft_consensus.cc:2454] T 2ddf84e9e05448098a356fddfa659b96 P 9f1437b726864fe499094ac03563fe08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec5e0cf1f22e44fba4bd414b63c9cad0 in term 1.
I20260929 10:50:07.806010 23133 raft_consensus.cc:2781] T 2ddf84e9e05448098a356fddfa659b96 P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:07.806056 23133 raft_consensus.cc:686] T 2ddf84e9e05448098a356fddfa659b96 P ec5e0cf1f22e44fba4bd414b63c9cad0 [term 1 LEADER]: Becoming Leader. State: Replica: ec5e0cf1f22e44fba4bd414b63c9cad0, State: Running, Role: LEADER
I20260929 10:50:07.806111 23133 consensus_queue.cc:237] T 2ddf84e9e05448098a356fddfa659b96 P ec5e0cf1f22e44fba4bd414b63c9cad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } }
I20260929 10:50:07.806789 22551 catalog_manager.cc:5495] T 2ddf84e9e05448098a356fddfa659b96 P ec5e0cf1f22e44fba4bd414b63c9cad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec5e0cf1f22e44fba4bd414b63c9cad0 (127.20.80.193). New cstate: current_term: 1 leader_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5e0cf1f22e44fba4bd414b63c9cad0" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 } health_report { overall_health: UNKNOWN } } }
I20260929 10:50:07.818126 23159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:07.816116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35430 (local address 127.20.80.193:34995)
0929 10:50:07.816174 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:07.816177 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:07.816210 (+ 33us) server_negotiation.cc:407] Connection header received
0929 10:50:07.816234 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:07.816235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:07.816243 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:07.816281 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:50:07.816374 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:07.816827 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:07.817612 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:07.817798 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:07.817854 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:07.817963 (+ 109us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:50:07.817989 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:50:07.818026 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:07.818056 (+ 30us) server_negotiation.cc:299] Negotiation successful
0929 10:50:07.818074 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260929 10:50:07.820134 22848 raft_consensus.cc:1261] T 2ddf84e9e05448098a356fddfa659b96 P bb91879b1a8d4c38b9af948109cafe10 [term 1 FOLLOWER]: Refusing update from remote peer ec5e0cf1f22e44fba4bd414b63c9cad0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:07.820171 22988 raft_consensus.cc:1261] T 2ddf84e9e05448098a356fddfa659b96 P 9f1437b726864fe499094ac03563fe08 [term 1 FOLLOWER]: Refusing update from remote peer ec5e0cf1f22e44fba4bd414b63c9cad0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:07.820353 23133 consensus_queue.cc:1035] T 2ddf84e9e05448098a356fddfa659b96 P ec5e0cf1f22e44fba4bd414b63c9cad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1437b726864fe499094ac03563fe08" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:50:07.820513 23116 consensus_queue.cc:1035] T 2ddf84e9e05448098a356fddfa659b96 P ec5e0cf1f22e44fba4bd414b63c9cad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb91879b1a8d4c38b9af948109cafe10" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 35241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 10:50:07.824649 23121 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260929 10:50:09.341132 22549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:50:09.341248 22549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:50:09.342018 22549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0c45c4ef55f34833825183d938c0215c: Sending DeleteTablet for 3 replicas of tablet 2ddf84e9e05448098a356fddfa659b96
I20260929 10:50:09.342311 22688 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ddf84e9e05448098a356fddfa659b96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:09 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:35410
I20260929 10:50:09.342363 22828 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ddf84e9e05448098a356fddfa659b96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:09 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:35996
I20260929 10:50:09.342434 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 22623
I20260929 10:50:09.342511 22968 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ddf84e9e05448098a356fddfa659b96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:09 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:58792
I20260929 10:50:09.343852 23203 tablet_replica.cc:330] T 2ddf84e9e05448098a356fddfa659b96 P bb91879b1a8d4c38b9af948109cafe10: stopping tablet replica
I20260929 10:50:09.343915 23203 raft_consensus.cc:2229] T 2ddf84e9e05448098a356fddfa659b96 P bb91879b1a8d4c38b9af948109cafe10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:50:09.343954 23203 raft_consensus.cc:2258] T 2ddf84e9e05448098a356fddfa659b96 P bb91879b1a8d4c38b9af948109cafe10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:09.344185 23203 ts_tablet_manager.cc:1905] T 2ddf84e9e05448098a356fddfa659b96 P bb91879b1a8d4c38b9af948109cafe10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:09.344635 23204 tablet_replica.cc:330] T 2ddf84e9e05448098a356fddfa659b96 P 9f1437b726864fe499094ac03563fe08: stopping tablet replica
I20260929 10:50:09.344694 23204 raft_consensus.cc:2229] T 2ddf84e9e05448098a356fddfa659b96 P 9f1437b726864fe499094ac03563fe08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:50:09.344726 23204 raft_consensus.cc:2258] T 2ddf84e9e05448098a356fddfa659b96 P 9f1437b726864fe499094ac03563fe08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:09.345003 23204 ts_tablet_manager.cc:1905] T 2ddf84e9e05448098a356fddfa659b96 P 9f1437b726864fe499094ac03563fe08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:09.346045 23203 ts_tablet_manager.cc:1918] T 2ddf84e9e05448098a356fddfa659b96 P bb91879b1a8d4c38b9af948109cafe10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 10:50:09.346107 23203 log.cc:1196] T 2ddf84e9e05448098a356fddfa659b96 P bb91879b1a8d4c38b9af948109cafe10: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-1/wal/wals/2ddf84e9e05448098a356fddfa659b96
I20260929 10:50:09.346321 23203 ts_tablet_manager.cc:1939] T 2ddf84e9e05448098a356fddfa659b96 P bb91879b1a8d4c38b9af948109cafe10: Deleting consensus metadata
I20260929 10:50:09.346686 23204 ts_tablet_manager.cc:1918] T 2ddf84e9e05448098a356fddfa659b96 P 9f1437b726864fe499094ac03563fe08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 10:50:09.346763 23204 log.cc:1196] T 2ddf84e9e05448098a356fddfa659b96 P 9f1437b726864fe499094ac03563fe08: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790678996126930-20803-0/minicluster-data/ts-2/wal/wals/2ddf84e9e05448098a356fddfa659b96
I20260929 10:50:09.346756 22536 catalog_manager.cc:4841] TS bb91879b1a8d4c38b9af948109cafe10 (127.20.80.194:35241): tablet 2ddf84e9e05448098a356fddfa659b96 (table test-table [id=c329eb12e01644cb93bb01a8d2b7c0b3]) successfully deleted
I20260929 10:50:09.346990 23204 ts_tablet_manager.cc:1939] T 2ddf84e9e05448098a356fddfa659b96 P 9f1437b726864fe499094ac03563fe08: Deleting consensus metadata
I20260929 10:50:09.347401 22535 catalog_manager.cc:4841] TS 9f1437b726864fe499094ac03563fe08 (127.20.80.195:41207): tablet 2ddf84e9e05448098a356fddfa659b96 (table test-table [id=c329eb12e01644cb93bb01a8d2b7c0b3]) successfully deleted
W20260929 10:50:09.351533 22538 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.80.193:34995 (error 108)
I20260929 10:50:09.351815 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 22763
I20260929 10:50:09.353349 23206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:09.351962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:34995 (local address 127.0.0.1:35432)
0929 10:50:09.353271 (+ 1309us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:09.353314 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.193:34995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1283,"thread_start_us":57,"threads_started":1}
W20260929 10:50:09.353467 22538 catalog_manager.cc:4584] TS ec5e0cf1f22e44fba4bd414b63c9cad0 (127.20.80.193:34995): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2ddf84e9e05448098a356fddfa659b96: Network error: Client connection negotiation failed: client connection to 127.20.80.193:34995: connect: Connection refused (error 111)
I20260929 10:50:09.358062 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 22903
I20260929 10:50:09.364630 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 22519
2026-09-29T10:50:09Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5774 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:50:09 dist-test-slave-vqch krb5kdc[23209](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:50:09 dist-test-slave-vqch krb5kdc[23209](info): set up 2 sockets
Sep 29 10:50:09 dist-test-slave-vqch krb5kdc[23209](info): commencing operation
krb5kdc: starting...
W20260929 10:50:10.412407 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:50:10 dist-test-slave-vqch krb5kdc[23209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:50:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:50:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.80.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
I20260929 10:50:10.531976 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:39001
--webserver_interface=127.20.80.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:46345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:39001
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:10.612995 23222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:10.613215 23222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:10.613241 23222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:10.614370 23222 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:50:10.614419 23222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:10.614436 23222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:10.614451 23222 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:50:10.614464 23222 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:50:10.615988 23222 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:46345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:39001
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:39001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:10.616272 23222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:10.616552 23222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 10:50:10.621872 23222 server_base.cc:1034] running on GCE node
W20260929 10:50:10.621935 23227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:10.621886 23228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:10.622133 23230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:10.622433 23222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:10.622733 23222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:10.623917 23222 hybrid_clock.cc:648] HybridClock initialized: now 1790679010623900 us; error 26 us; skew 500 ppm
Sep 29 10:50:10 dist-test-slave-vqch krb5kdc[23209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679010, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:10.626525 23222 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:50:10.626984 23222 webserver.cc:460] Webserver started at http://127.20.80.254:33855/ using document root <none> and password file <none>
I20260929 10:50:10.627228 23222 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:10.627300 23222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:10.627421 23222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:10.628496 23222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "408c44830d0b48869e24fa7841cc15b5"
format_stamp: "Formatted at 2026-09-29 10:50:10 on dist-test-slave-vqch"
server_key: "311cf58ab9ccdfeaca6c6f7526767454"
server_key_iv: "909afe762bec6ae7026ed647fa66ab04"
server_key_version: "encryptionkey@0"
I20260929 10:50:10.628849 23222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "408c44830d0b48869e24fa7841cc15b5"
format_stamp: "Formatted at 2026-09-29 10:50:10 on dist-test-slave-vqch"
server_key: "311cf58ab9ccdfeaca6c6f7526767454"
server_key_iv: "909afe762bec6ae7026ed647fa66ab04"
server_key_version: "encryptionkey@0"
I20260929 10:50:10.630317 23222 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:50:10.631075 23237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:10.631311 23222 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 10:50:10.631409 23222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "408c44830d0b48869e24fa7841cc15b5"
format_stamp: "Formatted at 2026-09-29 10:50:10 on dist-test-slave-vqch"
server_key: "311cf58ab9ccdfeaca6c6f7526767454"
server_key_iv: "909afe762bec6ae7026ed647fa66ab04"
server_key_version: "encryptionkey@0"
I20260929 10:50:10.631484 23222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:10.646409 23222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:10.648633 23222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:10.648780 23222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:10.652627 23222 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:39001
I20260929 10:50:10.652710 23289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:39001 every 8 connection(s)
I20260929 10:50:10.653117 23222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:50:10.653944 23290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:10.656225 23290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 408c44830d0b48869e24fa7841cc15b5: Bootstrap starting.
I20260929 10:50:10.656762 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 23222
I20260929 10:50:10.656860 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:50:10.656916 23290 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 408c44830d0b48869e24fa7841cc15b5: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:10.657076 20803 external_mini_cluster.cc:1428] Setting key 1b36dfa093e6f5c0e046455f0c5c5e7e
I20260929 10:50:10.657284 23290 log.cc:824] T 00000000000000000000000000000000 P 408c44830d0b48869e24fa7841cc15b5: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:10.658079 23290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 408c44830d0b48869e24fa7841cc15b5: No bootstrap required, opened a new log
I20260929 10:50:10.660542 23290 raft_consensus.cc:348] T 00000000000000000000000000000000 P 408c44830d0b48869e24fa7841cc15b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408c44830d0b48869e24fa7841cc15b5" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 39001 } }
I20260929 10:50:10.660708 23290 raft_consensus.cc:374] T 00000000000000000000000000000000 P 408c44830d0b48869e24fa7841cc15b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:10.660748 23290 raft_consensus.cc:729] T 00000000000000000000000000000000 P 408c44830d0b48869e24fa7841cc15b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 408c44830d0b48869e24fa7841cc15b5, State: Initialized, Role: FOLLOWER
I20260929 10:50:10.660890 23290 consensus_queue.cc:260] T 00000000000000000000000000000000 P 408c44830d0b48869e24fa7841cc15b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408c44830d0b48869e24fa7841cc15b5" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 39001 } }
I20260929 10:50:10.661006 23290 raft_consensus.cc:388] T 00000000000000000000000000000000 P 408c44830d0b48869e24fa7841cc15b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:50:10.661053 23290 raft_consensus.cc:482] T 00000000000000000000000000000000 P 408c44830d0b48869e24fa7841cc15b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:50:10.661103 23290 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 408c44830d0b48869e24fa7841cc15b5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:10.661688 23290 raft_consensus.cc:504] T 00000000000000000000000000000000 P 408c44830d0b48869e24fa7841cc15b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408c44830d0b48869e24fa7841cc15b5" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 39001 } }
I20260929 10:50:10.661788 23290 leader_election.cc:302] T 00000000000000000000000000000000 P 408c44830d0b48869e24fa7841cc15b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 408c44830d0b48869e24fa7841cc15b5; no voters:
I20260929 10:50:10.661953 23290 leader_election.cc:288] T 00000000000000000000000000000000 P 408c44830d0b48869e24fa7841cc15b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:50:10.662042 23295 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 408c44830d0b48869e24fa7841cc15b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:10.662194 23290 sys_catalog.cc:564] T 00000000000000000000000000000000 P 408c44830d0b48869e24fa7841cc15b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:50:10.662226 23295 raft_consensus.cc:686] T 00000000000000000000000000000000 P 408c44830d0b48869e24fa7841cc15b5 [term 1 LEADER]: Becoming Leader. State: Replica: 408c44830d0b48869e24fa7841cc15b5, State: Running, Role: LEADER
I20260929 10:50:10.663741 23295 consensus_queue.cc:237] T 00000000000000000000000000000000 P 408c44830d0b48869e24fa7841cc15b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408c44830d0b48869e24fa7841cc15b5" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 39001 } }
I20260929 10:50:10.664330 23296 sys_catalog.cc:455] T 00000000000000000000000000000000 P 408c44830d0b48869e24fa7841cc15b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "408c44830d0b48869e24fa7841cc15b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408c44830d0b48869e24fa7841cc15b5" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 39001 } } }
I20260929 10:50:10.664527 23296 sys_catalog.cc:458] T 00000000000000000000000000000000 P 408c44830d0b48869e24fa7841cc15b5 [sys.catalog]: This master's current role is: LEADER
Sep 29 10:50:10 dist-test-slave-vqch krb5kdc[23209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
W20260929 10:50:10.664714 23309 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 408c44830d0b48869e24fa7841cc15b5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:50:10.664757 23309 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:50:10.664774 23297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 408c44830d0b48869e24fa7841cc15b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 408c44830d0b48869e24fa7841cc15b5. Latest consensus state: current_term: 1 leader_uuid: "408c44830d0b48869e24fa7841cc15b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408c44830d0b48869e24fa7841cc15b5" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 39001 } } }
I20260929 10:50:10.664855 23297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 408c44830d0b48869e24fa7841cc15b5 [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:10.664937 23311 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:50:10.665153 23311 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:50:10.667346 23311 catalog_manager.cc:1347] Generated new cluster ID: 3d0ac5fa38c948c0b77c5b852f77c052
I20260929 10:50:10.667403 23311 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:50:10.668820 23294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:10.657563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34458 (local address 127.20.80.254:39001)
0929 10:50:10.657707 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:10.657716 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:10.657750 (+ 34us) server_negotiation.cc:407] Connection header received
0929 10:50:10.658341 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:10.658349 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:10.658430 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:10.658584 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:10.659240 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:10.660682 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:10.661574 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:10.662192 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:10.662635 (+ 443us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:10.665321 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:10.665325 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:10.665326 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:10.665337 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:10.667242 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:10.667621 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:10.667623 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:10.667624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:10.667730 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:10.667950 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:10.667950 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:10.667951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:10.668232 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:10.668364 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:10.668504 (+ 140us) server_negotiation.cc:299] Negotiation successful
0929 10:50:10.668623 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":68,"threads_started":1}
I20260929 10:50:10.676777 23311 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:50:10.677482 23311 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:50:10.690056 23311 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 408c44830d0b48869e24fa7841cc15b5: Generated new TSK 0
I20260929 10:50:10.690244 23311 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.80.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
I20260929 10:50:10.724468 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.193:0
--local_ip_for_outbound_sockets=127.20.80.193
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:39001
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:46345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:10.806244 23316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:10.806468 23316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:10.806493 23316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:10.807709 23316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:10.807751 23316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:10.807778 23316 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.193
I20260929 10:50:10.809386 23316 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:46345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:39001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.193
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:10.809634 23316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:10.809864 23316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:10.814996 23321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:10.815022 23316 server_base.cc:1034] running on GCE node
W20260929 10:50:10.815056 23324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:10.815006 23322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:10.815446 23316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:10.815702 23316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:10.816883 23316 hybrid_clock.cc:648] HybridClock initialized: now 1790679010816858 us; error 41 us; skew 500 ppm
Sep 29 10:50:10 dist-test-slave-vqch krb5kdc[23209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679010, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:10.819303 23316 init.cc:377] Logged in from keytab as kudu/127.20.80.193@KRBTEST.COM (short username kudu)
I20260929 10:50:10.819710 23316 webserver.cc:460] Webserver started at http://127.20.80.193:33541/ using document root <none> and password file <none>
I20260929 10:50:10.819938 23316 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:10.820003 23316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:10.820145 23316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:10.821256 23316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-0/data/instance:
uuid: "319ae3c7ce134a8a9ebc61d6aa70fc3a"
format_stamp: "Formatted at 2026-09-29 10:50:10 on dist-test-slave-vqch"
server_key: "e57fea3b63fdf99e202647d5432a5001"
server_key_iv: "914ea44a914b4e3b9902ca7177b9f2c7"
server_key_version: "encryptionkey@0"
I20260929 10:50:10.821605 23316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance:
uuid: "319ae3c7ce134a8a9ebc61d6aa70fc3a"
format_stamp: "Formatted at 2026-09-29 10:50:10 on dist-test-slave-vqch"
server_key: "e57fea3b63fdf99e202647d5432a5001"
server_key_iv: "914ea44a914b4e3b9902ca7177b9f2c7"
server_key_version: "encryptionkey@0"
I20260929 10:50:10.822909 23316 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:50:10.823611 23331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:10.823804 23316 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:10.823915 23316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-0/wal
uuid: "319ae3c7ce134a8a9ebc61d6aa70fc3a"
format_stamp: "Formatted at 2026-09-29 10:50:10 on dist-test-slave-vqch"
server_key: "e57fea3b63fdf99e202647d5432a5001"
server_key_iv: "914ea44a914b4e3b9902ca7177b9f2c7"
server_key_version: "encryptionkey@0"
I20260929 10:50:10.823987 23316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:10.832556 23316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:10.834653 23316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:10.834820 23316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:10.835021 23316 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:10.835316 23316 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:10.835374 23316 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:10.835417 23316 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:10.835443 23316 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:10.840801 23316 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.193:38413
I20260929 10:50:10.840870 23444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.193:38413 every 8 connection(s)
I20260929 10:50:10.841185 23316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
Sep 29 10:50:10 dist-test-slave-vqch krb5kdc[23209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679010, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:10.849153 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 23316
I20260929 10:50:10.849246 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance
I20260929 10:50:10.849429 20803 external_mini_cluster.cc:1428] Setting key cf55c01149d7d3b40a0c6dff69007a2b
I20260929 10:50:10.849805 23294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:10.841934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:33797 (local address 127.20.80.254:39001)
0929 10:50:10.841998 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:10.842000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:10.842147 (+ 147us) server_negotiation.cc:407] Connection header received
0929 10:50:10.842620 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:10.842621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:10.842631 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:10.842675 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:10.843543 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:10.844046 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:10.844956 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:10.845207 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:10.845259 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:10.847215 (+ 1956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:10.847216 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:10.847216 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:10.847223 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:10.848402 (+ 1179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:10.848741 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:10.848742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:10.848742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:10.848813 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:10.849088 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:10.849090 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:10.849090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:10.849237 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:10.849323 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:10.849619 (+ 296us) server_negotiation.cc:299] Negotiation successful
0929 10:50:10.849713 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 10:50:10.849993 23447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:10.841865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39001 (local address 127.20.80.193:33797)
0929 10:50:10.842021 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:10.842062 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:10.842525 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:10.842726 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:10.842730 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:10.843278 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:10.843465 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:10.843469 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:10.844120 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:10.844121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:10.844881 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:10.844884 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:10.845287 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:10.845288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:10.845388 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:10.845791 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:10.845799 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:10.847136 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:10.848470 (+ 1334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:10.848476 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:10.848478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:10.848681 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:10.848879 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:10.848879 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:10.848880 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:10.848970 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:10.849316 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:10.849323 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:10.849449 (+ 126us) client_negotiation.cc:769] Sending connection context
0929 10:50:10.849591 (+ 142us) client_negotiation.cc:240] Negotiation successful
0929 10:50:10.849747 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":89,"threads_started":1}
I20260929 10:50:10.850606 23445 heartbeater.cc:347] Connected to a master server at 127.20.80.254:39001
I20260929 10:50:10.850714 23445 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:10.850894 23445 heartbeater.cc:511] Master 127.20.80.254:39001 requested a full tablet report, sending...
I20260929 10:50:10.851418 23254 ts_manager.cc:194] Registered new tserver with Master: 319ae3c7ce134a8a9ebc61d6aa70fc3a (127.20.80.193:38413)
I20260929 10:50:10.852041 23254 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.193@KRBTEST.COM'} at 127.20.80.193:33797
WARNING: no policy specified for kudu/127.20.80.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
I20260929 10:50:10.876266 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.194:0
--local_ip_for_outbound_sockets=127.20.80.194
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:39001
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:46345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:10.956670 23450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:10.956928 23450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:10.956967 23450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:10.958128 23450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:10.958196 23450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:10.958238 23450 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.194
I20260929 10:50:10.959760 23450 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:46345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:39001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.194
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:10.960011 23450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:10.960237 23450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:10.965382 23455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:10.965391 23456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:10.965433 23458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:10.965540 23450 server_base.cc:1034] running on GCE node
I20260929 10:50:10.965744 23450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:10.965988 23450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:10.967139 23450 hybrid_clock.cc:648] HybridClock initialized: now 1790679010967127 us; error 30 us; skew 500 ppm
Sep 29 10:50:10 dist-test-slave-vqch krb5kdc[23209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679010, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:10.969563 23450 init.cc:377] Logged in from keytab as kudu/127.20.80.194@KRBTEST.COM (short username kudu)
I20260929 10:50:10.969980 23450 webserver.cc:460] Webserver started at http://127.20.80.194:36121/ using document root <none> and password file <none>
I20260929 10:50:10.970198 23450 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:10.970281 23450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:10.970419 23450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:10.971454 23450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-1/data/instance:
uuid: "dc3f2142aef6423d8506b4d7d28698a0"
format_stamp: "Formatted at 2026-09-29 10:50:10 on dist-test-slave-vqch"
server_key: "5fe30779df15d9cbbb85828952d6ef32"
server_key_iv: "bea272d9244dac1711806ff85935da8c"
server_key_version: "encryptionkey@0"
I20260929 10:50:10.971808 23450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance:
uuid: "dc3f2142aef6423d8506b4d7d28698a0"
format_stamp: "Formatted at 2026-09-29 10:50:10 on dist-test-slave-vqch"
server_key: "5fe30779df15d9cbbb85828952d6ef32"
server_key_iv: "bea272d9244dac1711806ff85935da8c"
server_key_version: "encryptionkey@0"
I20260929 10:50:10.973212 23450 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:50:10.973984 23465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:10.974174 23450 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:10.974264 23450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-1/wal
uuid: "dc3f2142aef6423d8506b4d7d28698a0"
format_stamp: "Formatted at 2026-09-29 10:50:10 on dist-test-slave-vqch"
server_key: "5fe30779df15d9cbbb85828952d6ef32"
server_key_iv: "bea272d9244dac1711806ff85935da8c"
server_key_version: "encryptionkey@0"
I20260929 10:50:10.974344 23450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:10.989455 23450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:10.991501 23450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:10.991611 23450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:10.991784 23450 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:10.992087 23450 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:10.992139 23450 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:10.992180 23450 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:10.992208 23450 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:10.997859 23450 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.194:42107
I20260929 10:50:10.997944 23578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.194:42107 every 8 connection(s)
I20260929 10:50:10.998238 23450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
I20260929 10:50:11.000816 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 23450
I20260929 10:50:11.000898 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance
I20260929 10:50:11.001112 20803 external_mini_cluster.cc:1428] Setting key 75c92d53f53ff3e191afa8a378fcc518
Sep 29 10:50:10 dist-test-slave-vqch krb5kdc[23209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679010, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:11.008414 23294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:10.998943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:52389 (local address 127.20.80.254:39001)
0929 10:50:10.998999 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:10.999001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:10.999214 (+ 213us) server_negotiation.cc:407] Connection header received
0929 10:50:10.999753 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:10.999754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:10.999766 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:10.999811 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:11.000756 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:11.001517 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:11.002580 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:11.002822 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:11.002899 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:11.005372 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:11.005374 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:11.005374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:11.005383 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:11.007069 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:11.007442 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:11.007443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:11.007444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:11.007523 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:11.007749 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:11.007750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:11.007751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:11.007897 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:11.007972 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:11.008205 (+ 233us) server_negotiation.cc:299] Negotiation successful
0929 10:50:11.008292 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260929 10:50:11.008432 23581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:10.998931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39001 (local address 127.20.80.194:52389)
0929 10:50:10.999106 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:10.999140 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:10.999646 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:10.999886 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:10.999890 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:11.000450 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:11.000644 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:11.000649 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:11.001602 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:11.001603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:11.002486 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:11.002489 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:11.002926 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:11.002928 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:11.003088 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:11.003616 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:11.003624 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:11.005279 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:11.007150 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:11.007156 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:11.007158 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:11.007365 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:11.007591 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:11.007592 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:11.007592 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:11.007689 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:11.007963 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:11.007967 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:11.008076 (+ 109us) client_negotiation.cc:769] Sending connection context
0929 10:50:11.008157 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 10:50:11.008271 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":91,"threads_started":1}
I20260929 10:50:11.009021 23579 heartbeater.cc:347] Connected to a master server at 127.20.80.254:39001
I20260929 10:50:11.009128 23579 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:11.009305 23579 heartbeater.cc:511] Master 127.20.80.254:39001 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.20.80.195@KRBTEST.COM; defaulting to no policy
I20260929 10:50:11.009676 23254 ts_manager.cc:194] Registered new tserver with Master: dc3f2142aef6423d8506b4d7d28698a0 (127.20.80.194:42107)
I20260929 10:50:11.010056 23254 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.194@KRBTEST.COM'} at 127.20.80.194:52389
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
I20260929 10:50:11.027987 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.195:0
--local_ip_for_outbound_sockets=127.20.80.195
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:39001
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:46345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:11.107815 23584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:11.108009 23584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:11.108033 23584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:11.109148 23584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:11.109179 23584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:11.109205 23584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.195
I20260929 10:50:11.110534 23584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:46345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:39001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.195
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:11.110749 23584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:11.110927 23584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:11.115775 23589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:11.115809 23590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:11.115972 23584 server_base.cc:1034] running on GCE node
W20260929 10:50:11.115844 23592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:11.116225 23584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:11.116425 23584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:11.117580 23584 hybrid_clock.cc:648] HybridClock initialized: now 1790679011117563 us; error 29 us; skew 500 ppm
Sep 29 10:50:11 dist-test-slave-vqch krb5kdc[23209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679011, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:11.119957 23584 init.cc:377] Logged in from keytab as kudu/127.20.80.195@KRBTEST.COM (short username kudu)
I20260929 10:50:11.120396 23584 webserver.cc:460] Webserver started at http://127.20.80.195:42005/ using document root <none> and password file <none>
I20260929 10:50:11.120587 23584 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:11.120625 23584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:11.120710 23584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:11.121767 23584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-2/data/instance:
uuid: "346a1db999fe4788978b6ed5f28ce7f7"
format_stamp: "Formatted at 2026-09-29 10:50:11 on dist-test-slave-vqch"
server_key: "a97f473e0e257324909df01edcbad4a4"
server_key_iv: "05ff7e3f6dc006f3f85d41b0a543f646"
server_key_version: "encryptionkey@0"
I20260929 10:50:11.122095 23584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance:
uuid: "346a1db999fe4788978b6ed5f28ce7f7"
format_stamp: "Formatted at 2026-09-29 10:50:11 on dist-test-slave-vqch"
server_key: "a97f473e0e257324909df01edcbad4a4"
server_key_iv: "05ff7e3f6dc006f3f85d41b0a543f646"
server_key_version: "encryptionkey@0"
I20260929 10:50:11.123349 23584 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:50:11.123977 23599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:11.124153 23584 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:11.124228 23584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-2/wal
uuid: "346a1db999fe4788978b6ed5f28ce7f7"
format_stamp: "Formatted at 2026-09-29 10:50:11 on dist-test-slave-vqch"
server_key: "a97f473e0e257324909df01edcbad4a4"
server_key_iv: "05ff7e3f6dc006f3f85d41b0a543f646"
server_key_version: "encryptionkey@0"
I20260929 10:50:11.124280 23584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:11.141243 23584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:11.143226 23584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:11.143337 23584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:11.143527 23584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:11.143841 23584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:11.143878 23584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:11.143901 23584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:11.143949 23584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:11.149613 23584 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.195:43919
I20260929 10:50:11.149711 23712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.195:43919 every 8 connection(s)
I20260929 10:50:11.150024 23584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
I20260929 10:50:11.152611 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 23584
I20260929 10:50:11.152742 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance
I20260929 10:50:11.152915 20803 external_mini_cluster.cc:1428] Setting key 83556d14240f590ebab7da34f690fe8e
Sep 29 10:50:11 dist-test-slave-vqch krb5kdc[23209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679011, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:11.160000 23294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:11.150883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:34571 (local address 127.20.80.254:39001)
0929 10:50:11.150954 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:11.150957 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:11.151214 (+ 257us) server_negotiation.cc:407] Connection header received
0929 10:50:11.151698 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:11.151699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:11.151712 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:11.151776 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:11.152750 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:11.153563 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:11.154719 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:11.154919 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:11.154978 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:11.157211 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:11.157212 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:11.157212 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:11.157219 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:11.158586 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:11.159104 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:11.159104 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:11.159105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:11.159152 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:11.159373 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:11.159373 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:11.159374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:11.159471 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:11.159521 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:11.159810 (+ 289us) server_negotiation.cc:299] Negotiation successful
0929 10:50:11.159897 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 10:50:11.160216 23715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:11.150872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39001 (local address 127.20.80.195:34571)
0929 10:50:11.151080 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:11.151117 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:11.151587 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:11.151848 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:11.151851 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:11.152453 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:11.152648 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:11.152653 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:11.153691 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:11.153692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:11.154632 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:11.154635 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:11.155012 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:11.155013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:11.155136 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:11.155626 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:11.155635 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:11.157103 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:11.158800 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:11.158806 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:11.158808 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:11.159034 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:11.159207 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:11.159208 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:11.159208 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:11.159303 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:11.159524 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:11.159528 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:11.159644 (+ 116us) client_negotiation.cc:769] Sending connection context
0929 10:50:11.159781 (+ 137us) client_negotiation.cc:240] Negotiation successful
0929 10:50:11.159978 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":106,"threads_started":1}
I20260929 10:50:11.160686 23713 heartbeater.cc:347] Connected to a master server at 127.20.80.254:39001
I20260929 10:50:11.160781 23713 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:11.160971 23713 heartbeater.cc:511] Master 127.20.80.254:39001 requested a full tablet report, sending...
I20260929 10:50:11.161399 23254 ts_manager.cc:194] Registered new tserver with Master: 346a1db999fe4788978b6ed5f28ce7f7 (127.20.80.195:43919)
I20260929 10:50:11.161788 23254 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.195@KRBTEST.COM'} at 127.20.80.195:34571
I20260929 10:50:11.164661 20803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 10:50:11.175462 23294 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:11.174530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34470 (local address 127.20.80.254:39001)
0929 10:50:11.174603 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:11.174606 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:11.174702 (+ 96us) server_negotiation.cc:407] Connection header received
0929 10:50:11.174807 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:11.174808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:11.174819 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:11.174882 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:11.175392 (+ 510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34470: BlockingRecv error: recv got EOF from 127.0.0.1:34470 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260929 10:50:11.176365 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 23316
I20260929 10:50:11.181944 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 23450
I20260929 10:50:11.186852 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 23584
I20260929 10:50:11.191390 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 23222
2026-09-29T10:50:11Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1824 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:50:11 dist-test-slave-vqch krb5kdc[23725](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:50:11 dist-test-slave-vqch krb5kdc[23725](info): set up 2 sockets
Sep 29 10:50:11 dist-test-slave-vqch krb5kdc[23725](info): commencing operation
krb5kdc: starting...
W20260929 10:50:12.260540 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:50:12 dist-test-slave-vqch krb5kdc[23725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:50:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:50:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.80.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
I20260929 10:50:12.376067 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:33665
--webserver_interface=127.20.80.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:40665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:33665
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:12.455374 23738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:12.455565 23738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:12.455587 23738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:12.456657 23738 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:50:12.456691 23738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:12.456705 23738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:12.456717 23738 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:50:12.456729 23738 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:50:12.458168 23738 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:40665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:33665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:33665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:12.458351 23738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:12.458523 23738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:12.463397 23744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:12.463341 23743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:12.463518 23746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:12.463593 23738 server_base.cc:1034] running on GCE node
I20260929 10:50:12.463843 23738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:12.464097 23738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:12.465229 23738 hybrid_clock.cc:648] HybridClock initialized: now 1790679012465207 us; error 41 us; skew 500 ppm
Sep 29 10:50:12 dist-test-slave-vqch krb5kdc[23725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679012, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:12.467643 23738 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:50:12.468025 23738 webserver.cc:460] Webserver started at http://127.20.80.254:43595/ using document root <none> and password file <none>
I20260929 10:50:12.468223 23738 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:12.468263 23738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:12.468349 23738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:12.469419 23738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "6dc6af5634884ceb82a364a6f57ab0c7"
format_stamp: "Formatted at 2026-09-29 10:50:12 on dist-test-slave-vqch"
server_key: "8d9fd6a7cdb79a2ed0a1f4fb0c73bf20"
server_key_iv: "a8c5b792fcb81fa411f3ec33506719cf"
server_key_version: "encryptionkey@0"
I20260929 10:50:12.469717 23738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "6dc6af5634884ceb82a364a6f57ab0c7"
format_stamp: "Formatted at 2026-09-29 10:50:12 on dist-test-slave-vqch"
server_key: "8d9fd6a7cdb79a2ed0a1f4fb0c73bf20"
server_key_iv: "a8c5b792fcb81fa411f3ec33506719cf"
server_key_version: "encryptionkey@0"
I20260929 10:50:12.471058 23738 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:50:12.471825 23753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:12.472035 23738 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:12.472131 23738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "6dc6af5634884ceb82a364a6f57ab0c7"
format_stamp: "Formatted at 2026-09-29 10:50:12 on dist-test-slave-vqch"
server_key: "8d9fd6a7cdb79a2ed0a1f4fb0c73bf20"
server_key_iv: "a8c5b792fcb81fa411f3ec33506719cf"
server_key_version: "encryptionkey@0"
I20260929 10:50:12.472208 23738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:12.509837 23738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:12.512051 23738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:12.512184 23738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:12.516183 23738 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:33665
I20260929 10:50:12.516240 23805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:33665 every 8 connection(s)
I20260929 10:50:12.516716 23738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:50:12.517419 23806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:12.519738 23806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dc6af5634884ceb82a364a6f57ab0c7: Bootstrap starting.
I20260929 10:50:12.520330 23806 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6dc6af5634884ceb82a364a6f57ab0c7: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:12.520630 23806 log.cc:824] T 00000000000000000000000000000000 P 6dc6af5634884ceb82a364a6f57ab0c7: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:12.520924 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 23738
I20260929 10:50:12.521062 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:50:12.521257 20803 external_mini_cluster.cc:1428] Setting key a7b5fc8de79db004fa8bded12659950a
I20260929 10:50:12.521425 23806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dc6af5634884ceb82a364a6f57ab0c7: No bootstrap required, opened a new log
I20260929 10:50:12.524250 23806 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6dc6af5634884ceb82a364a6f57ab0c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dc6af5634884ceb82a364a6f57ab0c7" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 33665 } }
I20260929 10:50:12.524403 23806 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6dc6af5634884ceb82a364a6f57ab0c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:12.524451 23806 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6dc6af5634884ceb82a364a6f57ab0c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc6af5634884ceb82a364a6f57ab0c7, State: Initialized, Role: FOLLOWER
I20260929 10:50:12.524576 23806 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6dc6af5634884ceb82a364a6f57ab0c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dc6af5634884ceb82a364a6f57ab0c7" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 33665 } }
I20260929 10:50:12.524664 23806 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6dc6af5634884ceb82a364a6f57ab0c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:50:12.524708 23806 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6dc6af5634884ceb82a364a6f57ab0c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:50:12.524757 23806 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6dc6af5634884ceb82a364a6f57ab0c7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:12.525457 23806 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6dc6af5634884ceb82a364a6f57ab0c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dc6af5634884ceb82a364a6f57ab0c7" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 33665 } }
I20260929 10:50:12.525568 23806 leader_election.cc:302] T 00000000000000000000000000000000 P 6dc6af5634884ceb82a364a6f57ab0c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6dc6af5634884ceb82a364a6f57ab0c7; no voters:
I20260929 10:50:12.525720 23806 leader_election.cc:288] T 00000000000000000000000000000000 P 6dc6af5634884ceb82a364a6f57ab0c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:50:12.525774 23811 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6dc6af5634884ceb82a364a6f57ab0c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:12.525928 23811 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6dc6af5634884ceb82a364a6f57ab0c7 [term 1 LEADER]: Becoming Leader. State: Replica: 6dc6af5634884ceb82a364a6f57ab0c7, State: Running, Role: LEADER
I20260929 10:50:12.525967 23806 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6dc6af5634884ceb82a364a6f57ab0c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:50:12.526087 23811 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6dc6af5634884ceb82a364a6f57ab0c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dc6af5634884ceb82a364a6f57ab0c7" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 33665 } }
I20260929 10:50:12.526463 23813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dc6af5634884ceb82a364a6f57ab0c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6dc6af5634884ceb82a364a6f57ab0c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dc6af5634884ceb82a364a6f57ab0c7" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 33665 } } }
I20260929 10:50:12.526567 23813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dc6af5634884ceb82a364a6f57ab0c7 [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:12.527588 23812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dc6af5634884ceb82a364a6f57ab0c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6dc6af5634884ceb82a364a6f57ab0c7. Latest consensus state: current_term: 1 leader_uuid: "6dc6af5634884ceb82a364a6f57ab0c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dc6af5634884ceb82a364a6f57ab0c7" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 33665 } } }
I20260929 10:50:12.527674 23812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dc6af5634884ceb82a364a6f57ab0c7 [sys.catalog]: This master's current role is: LEADER
W20260929 10:50:12.527887 23827 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6dc6af5634884ceb82a364a6f57ab0c7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:50:12.527947 23827 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:50:12.528007 23820 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:50:12.528183 23820 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 29 10:50:12 dist-test-slave-vqch krb5kdc[23725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:12.529769 23820 catalog_manager.cc:1347] Generated new cluster ID: 6190529af6cf460da142785affb72169
I20260929 10:50:12.529889 23820 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:50:12.533345 23810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:12.521821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45116 (local address 127.20.80.254:33665)
0929 10:50:12.522023 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:12.522031 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:12.522059 (+ 28us) server_negotiation.cc:407] Connection header received
0929 10:50:12.522769 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:12.522777 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:12.522828 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:12.522949 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:12.523779 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:12.524803 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:12.525910 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:12.526571 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:12.526699 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:12.529728 (+ 3029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:12.529733 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:12.529733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:12.529745 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:12.531629 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:12.532045 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:12.532048 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:12.532048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:12.532172 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:12.532410 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:12.532411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:12.532411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:12.532665 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:12.532843 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:12.532944 (+ 101us) server_negotiation.cc:299] Negotiation successful
0929 10:50:12.533113 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":94,"threads_started":1}
I20260929 10:50:12.534255 20803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:50:12.535913 23820 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:50:12.536649 23820 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:50:12.552749 23810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:12.549339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45132 (local address 127.20.80.254:33665)
0929 10:50:12.549386 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:12.549388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:12.549507 (+ 119us) server_negotiation.cc:407] Connection header received
0929 10:50:12.549586 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:12.549587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:12.549597 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:12.549655 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:12.550001 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:12.550527 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:12.551273 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:12.551493 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:12.551577 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:12.552274 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:12.552275 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:12.552276 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:50:12.552467 (+ 191us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:12.552483 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:50:12.552489 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:12.552557 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:12.552624 (+ 67us) server_negotiation.cc:299] Negotiation successful
0929 10:50:12.552651 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 10:50:12.557662 23820 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6dc6af5634884ceb82a364a6f57ab0c7: Generated new TSK 0
I20260929 10:50:12.557840 23820 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:50:12.563321 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 23738
2026-09-29T10:50:12Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1370 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:50:12 dist-test-slave-vqch krb5kdc[23839](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:50:12 dist-test-slave-vqch krb5kdc[23839](info): set up 2 sockets
Sep 29 10:50:12 dist-test-slave-vqch krb5kdc[23839](info): commencing operation
krb5kdc: starting...
W20260929 10:50:13.588492 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:50:13 dist-test-slave-vqch krb5kdc[23839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:50:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:50:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.80.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
I20260929 10:50:13.711153 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:42107
--webserver_interface=127.20.80.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:43065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:42107
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:13.792445 23852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:13.792642 23852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:13.792671 23852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:13.793883 23852 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:50:13.793927 23852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:13.793941 23852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:13.793953 23852 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:50:13.793965 23852 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:50:13.795526 23852 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:43065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:42107
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:42107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:13.795778 23852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:13.796002 23852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:13.801040 23858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:13.801081 23857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:13.801118 23852 server_base.cc:1034] running on GCE node
W20260929 10:50:13.801062 23860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:13.801446 23852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:13.801733 23852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:13.802911 23852 hybrid_clock.cc:648] HybridClock initialized: now 1790679013802891 us; error 36 us; skew 500 ppm
Sep 29 10:50:13 dist-test-slave-vqch krb5kdc[23839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679013, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:13.810892 23852 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:50:13.811365 23852 webserver.cc:460] Webserver started at http://127.20.80.254:37631/ using document root <none> and password file <none>
I20260929 10:50:13.811594 23852 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:13.811655 23852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:13.811798 23852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:13.812916 23852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "ac3dd52b19e44a8588b37723041c88f6"
format_stamp: "Formatted at 2026-09-29 10:50:13 on dist-test-slave-vqch"
server_key: "8c04e4cf711acf7852f61821a06de326"
server_key_iv: "ff053ed5cad680b22df57ffee99e2e21"
server_key_version: "encryptionkey@0"
I20260929 10:50:13.813341 23852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "ac3dd52b19e44a8588b37723041c88f6"
format_stamp: "Formatted at 2026-09-29 10:50:13 on dist-test-slave-vqch"
server_key: "8c04e4cf711acf7852f61821a06de326"
server_key_iv: "ff053ed5cad680b22df57ffee99e2e21"
server_key_version: "encryptionkey@0"
I20260929 10:50:13.814661 23852 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:13.815410 23867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:13.815572 23852 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:13.815666 23852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "ac3dd52b19e44a8588b37723041c88f6"
format_stamp: "Formatted at 2026-09-29 10:50:13 on dist-test-slave-vqch"
server_key: "8c04e4cf711acf7852f61821a06de326"
server_key_iv: "ff053ed5cad680b22df57ffee99e2e21"
server_key_version: "encryptionkey@0"
I20260929 10:50:13.815742 23852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:13.852049 23852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:13.854172 23852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:13.854329 23852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:13.858428 23852 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:42107
I20260929 10:50:13.858474 23919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:42107 every 8 connection(s)
I20260929 10:50:13.858841 23852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:50:13.859825 23920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:13.862079 23920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac3dd52b19e44a8588b37723041c88f6: Bootstrap starting.
I20260929 10:50:13.862610 23920 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac3dd52b19e44a8588b37723041c88f6: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:13.862839 23920 log.cc:824] T 00000000000000000000000000000000 P ac3dd52b19e44a8588b37723041c88f6: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:13.863479 23920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac3dd52b19e44a8588b37723041c88f6: No bootstrap required, opened a new log
I20260929 10:50:13.865002 23920 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac3dd52b19e44a8588b37723041c88f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3dd52b19e44a8588b37723041c88f6" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 42107 } }
I20260929 10:50:13.865139 23920 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac3dd52b19e44a8588b37723041c88f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:13.865159 23920 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac3dd52b19e44a8588b37723041c88f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac3dd52b19e44a8588b37723041c88f6, State: Initialized, Role: FOLLOWER
I20260929 10:50:13.865304 23920 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac3dd52b19e44a8588b37723041c88f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3dd52b19e44a8588b37723041c88f6" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 42107 } }
I20260929 10:50:13.865393 23920 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac3dd52b19e44a8588b37723041c88f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:50:13.865417 23920 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac3dd52b19e44a8588b37723041c88f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:50:13.865451 23920 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac3dd52b19e44a8588b37723041c88f6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:13.866016 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 23852
I20260929 10:50:13.866132 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:50:13.866303 20803 external_mini_cluster.cc:1428] Setting key a62ecee55b30e55278dc320b8a47c90c
I20260929 10:50:13.866323 23920 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac3dd52b19e44a8588b37723041c88f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3dd52b19e44a8588b37723041c88f6" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 42107 } }
I20260929 10:50:13.866448 23920 leader_election.cc:302] T 00000000000000000000000000000000 P ac3dd52b19e44a8588b37723041c88f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac3dd52b19e44a8588b37723041c88f6; no voters:
I20260929 10:50:13.866621 23920 leader_election.cc:288] T 00000000000000000000000000000000 P ac3dd52b19e44a8588b37723041c88f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:50:13.866665 23923 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac3dd52b19e44a8588b37723041c88f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:13.866879 23923 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac3dd52b19e44a8588b37723041c88f6 [term 1 LEADER]: Becoming Leader. State: Replica: ac3dd52b19e44a8588b37723041c88f6, State: Running, Role: LEADER
I20260929 10:50:13.866950 23920 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac3dd52b19e44a8588b37723041c88f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:50:13.867026 23923 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac3dd52b19e44a8588b37723041c88f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3dd52b19e44a8588b37723041c88f6" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 42107 } }
I20260929 10:50:13.867343 23926 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac3dd52b19e44a8588b37723041c88f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac3dd52b19e44a8588b37723041c88f6. Latest consensus state: current_term: 1 leader_uuid: "ac3dd52b19e44a8588b37723041c88f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3dd52b19e44a8588b37723041c88f6" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 42107 } } }
I20260929 10:50:13.867348 23925 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac3dd52b19e44a8588b37723041c88f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac3dd52b19e44a8588b37723041c88f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3dd52b19e44a8588b37723041c88f6" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 42107 } } }
I20260929 10:50:13.867540 23926 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac3dd52b19e44a8588b37723041c88f6 [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:13.867554 23925 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac3dd52b19e44a8588b37723041c88f6 [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:13.868010 23935 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:50:13.868525 23935 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 29 10:50:13 dist-test-slave-vqch krb5kdc[23839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:13.873467 23935 catalog_manager.cc:1347] Generated new cluster ID: db0bd60fe9594ad086ec3e35afd484fa
I20260929 10:50:13.873515 23935 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:50:13.877094 23927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:13.866756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53280 (local address 127.20.80.254:42107)
0929 10:50:13.867701 (+ 945us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:13.867710 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:13.867894 (+ 184us) server_negotiation.cc:407] Connection header received
0929 10:50:13.868018 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:13.868025 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:13.868084 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:13.868220 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:13.869076 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:13.870192 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:13.870942 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:13.871364 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:13.871476 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:13.873927 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:13.873931 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:13.873931 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:13.873943 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:13.875563 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:13.875934 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:13.875937 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:13.875937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:13.876041 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:13.876243 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:13.876244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:13.876244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:13.876479 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:13.876637 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:13.876741 (+ 104us) server_negotiation.cc:299] Negotiation successful
0929 10:50:13.876877 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":886,"thread_start_us":106,"threads_started":1}
I20260929 10:50:13.901439 23935 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:50:13.902002 23935 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:50:13.903748 23935 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac3dd52b19e44a8588b37723041c88f6: Generated new TSK 0
I20260929 10:50:13.903923 23935 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.80.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
I20260929 10:50:13.939695 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.193:0
--local_ip_for_outbound_sockets=127.20.80.193
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:42107
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:43065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:14.020581 23946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:14.020784 23946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:14.020807 23946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:14.021973 23946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:14.022014 23946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:14.022042 23946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.193
I20260929 10:50:14.023475 23946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:43065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:42107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.193
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:14.023697 23946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:14.023914 23946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:14.029125 23954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:14.029132 23951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:14.029405 23946 server_base.cc:1034] running on GCE node
W20260929 10:50:14.029417 23952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:14.029805 23946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:14.030062 23946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:14.031221 23946 hybrid_clock.cc:648] HybridClock initialized: now 1790679014031199 us; error 38 us; skew 500 ppm
Sep 29 10:50:14 dist-test-slave-vqch krb5kdc[23839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679014, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:14.033735 23946 init.cc:377] Logged in from keytab as kudu/127.20.80.193@KRBTEST.COM (short username kudu)
I20260929 10:50:14.034211 23946 webserver.cc:460] Webserver started at http://127.20.80.193:43961/ using document root <none> and password file <none>
I20260929 10:50:14.034449 23946 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:14.034512 23946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:14.034667 23946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:14.035704 23946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-0/data/instance:
uuid: "057e13c750944611bdd015dec4720161"
format_stamp: "Formatted at 2026-09-29 10:50:14 on dist-test-slave-vqch"
server_key: "9a710cb9683cb3ff776f21eb5a6d2cdc"
server_key_iv: "e2709fcc7f6a17b12228312d56674f56"
server_key_version: "encryptionkey@0"
I20260929 10:50:14.036065 23946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance:
uuid: "057e13c750944611bdd015dec4720161"
format_stamp: "Formatted at 2026-09-29 10:50:14 on dist-test-slave-vqch"
server_key: "9a710cb9683cb3ff776f21eb5a6d2cdc"
server_key_iv: "e2709fcc7f6a17b12228312d56674f56"
server_key_version: "encryptionkey@0"
I20260929 10:50:14.037448 23946 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 10:50:14.038152 23961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:14.038307 23946 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:14.038399 23946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-0/wal
uuid: "057e13c750944611bdd015dec4720161"
format_stamp: "Formatted at 2026-09-29 10:50:14 on dist-test-slave-vqch"
server_key: "9a710cb9683cb3ff776f21eb5a6d2cdc"
server_key_iv: "e2709fcc7f6a17b12228312d56674f56"
server_key_version: "encryptionkey@0"
I20260929 10:50:14.038475 23946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:14.051460 23946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:14.053275 23946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:14.053422 23946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:14.053637 23946 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:14.053931 23946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:14.053979 23946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:14.054018 23946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:14.054045 23946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:14.059696 23946 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.193:38409
I20260929 10:50:14.059736 24075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.193:38409 every 8 connection(s)
I20260929 10:50:14.060065 23946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
I20260929 10:50:14.064725 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 23946
I20260929 10:50:14.064847 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance
I20260929 10:50:14.065104 20803 external_mini_cluster.cc:1428] Setting key b05b2693421699d55d450bc1704706f6
Sep 29 10:50:14 dist-test-slave-vqch krb5kdc[23839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679014, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:14.070022 23927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:14.060795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:52071 (local address 127.20.80.254:42107)
0929 10:50:14.060893 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:14.060895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:14.061050 (+ 155us) server_negotiation.cc:407] Connection header received
0929 10:50:14.061564 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:14.061565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:14.061580 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:14.061629 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:14.062640 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:14.063176 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:14.064137 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:14.064326 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:14.064388 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:14.066585 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:14.066586 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:14.066586 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:14.066594 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:14.068344 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:14.068873 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:14.068874 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:14.068875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:14.068938 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:14.069340 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:14.069341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:14.069342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:14.069455 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:14.069521 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:14.069844 (+ 323us) server_negotiation.cc:299] Negotiation successful
0929 10:50:14.069930 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 10:50:14.070259 24078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:14.060759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:42107 (local address 127.20.80.193:52071)
0929 10:50:14.060915 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:14.060951 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:14.061474 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:14.061691 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:14.061694 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:14.062264 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:14.062471 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:14.062475 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:14.063261 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:14.063262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:14.064061 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:14.064063 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:14.064401 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:14.064401 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:14.064497 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:14.065034 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:14.065044 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:14.066502 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:14.068417 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:14.068425 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:14.068427 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:14.068754 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:14.069044 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:14.069045 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:14.069045 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:14.069188 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:14.069530 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:14.069536 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:14.069699 (+ 163us) client_negotiation.cc:769] Sending connection context
0929 10:50:14.069809 (+ 110us) client_negotiation.cc:240] Negotiation successful
0929 10:50:14.069987 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":81,"threads_started":1}
I20260929 10:50:14.070977 24076 heartbeater.cc:347] Connected to a master server at 127.20.80.254:42107
I20260929 10:50:14.071070 24076 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:14.071256 24076 heartbeater.cc:511] Master 127.20.80.254:42107 requested a full tablet report, sending...
I20260929 10:50:14.071805 23884 ts_manager.cc:194] Registered new tserver with Master: 057e13c750944611bdd015dec4720161 (127.20.80.193:38409)
I20260929 10:50:14.072743 23884 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.193@KRBTEST.COM'} at 127.20.80.193:52071
WARNING: no policy specified for kudu/127.20.80.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
I20260929 10:50:14.091758 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.194:0
--local_ip_for_outbound_sockets=127.20.80.194
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:42107
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:43065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:14.171394 24081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:14.171589 24081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:14.171612 24081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:14.172693 24081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:14.172727 24081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:14.172753 24081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.194
I20260929 10:50:14.174129 24081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:43065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:42107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.194
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:14.174314 24081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:14.174474 24081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:14.179286 24089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:14.179303 24086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:14.179286 24087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:14.179418 24081 server_base.cc:1034] running on GCE node
I20260929 10:50:14.179661 24081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:14.179900 24081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:14.181031 24081 hybrid_clock.cc:648] HybridClock initialized: now 1790679014181017 us; error 41 us; skew 500 ppm
Sep 29 10:50:14 dist-test-slave-vqch krb5kdc[23839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679014, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:14.183377 24081 init.cc:377] Logged in from keytab as kudu/127.20.80.194@KRBTEST.COM (short username kudu)
I20260929 10:50:14.183820 24081 webserver.cc:460] Webserver started at http://127.20.80.194:40891/ using document root <none> and password file <none>
I20260929 10:50:14.184048 24081 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:14.184111 24081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:14.184242 24081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:14.185308 24081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-1/data/instance:
uuid: "bc3e1ae5a80e4b918fe2945300f58997"
format_stamp: "Formatted at 2026-09-29 10:50:14 on dist-test-slave-vqch"
server_key: "2428b98293dd69aba4f5dec6e5eb4f80"
server_key_iv: "fb3080a9ba68cede0e53f51102d4ef10"
server_key_version: "encryptionkey@0"
I20260929 10:50:14.185649 24081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance:
uuid: "bc3e1ae5a80e4b918fe2945300f58997"
format_stamp: "Formatted at 2026-09-29 10:50:14 on dist-test-slave-vqch"
server_key: "2428b98293dd69aba4f5dec6e5eb4f80"
server_key_iv: "fb3080a9ba68cede0e53f51102d4ef10"
server_key_version: "encryptionkey@0"
I20260929 10:50:14.186987 24081 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:50:14.187685 24096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:14.187902 24081 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:14.188009 24081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-1/wal
uuid: "bc3e1ae5a80e4b918fe2945300f58997"
format_stamp: "Formatted at 2026-09-29 10:50:14 on dist-test-slave-vqch"
server_key: "2428b98293dd69aba4f5dec6e5eb4f80"
server_key_iv: "fb3080a9ba68cede0e53f51102d4ef10"
server_key_version: "encryptionkey@0"
I20260929 10:50:14.188095 24081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:14.203302 24081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:14.205651 24081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:14.205806 24081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:14.206029 24081 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:14.206355 24081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:14.206406 24081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:14.206449 24081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:14.206475 24081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:14.212445 24081 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.194:32885
I20260929 10:50:14.212503 24211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.194:32885 every 8 connection(s)
I20260929 10:50:14.212823 24081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
I20260929 10:50:14.216369 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 24081
I20260929 10:50:14.216470 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance
I20260929 10:50:14.216638 20803 external_mini_cluster.cc:1428] Setting key 0e0293a8b9f743818edff4eccfc165aa
Sep 29 10:50:14 dist-test-slave-vqch krb5kdc[23839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679014, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:14.222653 23927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:14.213568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:38083 (local address 127.20.80.254:42107)
0929 10:50:14.213633 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:14.213635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:14.213820 (+ 185us) server_negotiation.cc:407] Connection header received
0929 10:50:14.214327 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:14.214329 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:14.214338 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:14.214383 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:14.215261 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:14.215773 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:14.217110 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:14.217398 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:14.217476 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:14.219656 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:14.219658 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:14.219658 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:14.219665 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:14.221271 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:14.221649 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:14.221650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:14.221651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:14.221719 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:14.222010 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:14.222011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:14.222011 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:14.222121 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:14.222176 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:14.222518 (+ 342us) server_negotiation.cc:299] Negotiation successful
0929 10:50:14.222590 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 10:50:14.222968 24214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:14.213538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:42107 (local address 127.20.80.194:38083)
0929 10:50:14.213701 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:14.213742 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:14.214220 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:14.214439 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:14.214442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:14.214999 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:14.215183 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:14.215188 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:14.215851 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:14.215852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:14.217019 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:14.217023 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:14.217479 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:14.217480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:14.217587 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:14.218135 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:14.218143 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:14.219565 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:14.221349 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:14.221354 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:14.221356 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:14.221569 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:14.221785 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:14.221786 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:14.221786 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:14.221937 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:14.222187 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:14.222194 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:14.222368 (+ 174us) client_negotiation.cc:769] Sending connection context
0929 10:50:14.222490 (+ 122us) client_negotiation.cc:240] Negotiation successful
0929 10:50:14.222686 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
I20260929 10:50:14.223423 24212 heartbeater.cc:347] Connected to a master server at 127.20.80.254:42107
I20260929 10:50:14.223529 24212 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:14.223714 24212 heartbeater.cc:511] Master 127.20.80.254:42107 requested a full tablet report, sending...
I20260929 10:50:14.224130 23884 ts_manager.cc:194] Registered new tserver with Master: bc3e1ae5a80e4b918fe2945300f58997 (127.20.80.194:32885)
I20260929 10:50:14.224516 23884 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.194@KRBTEST.COM'} at 127.20.80.194:38083
WARNING: no policy specified for kudu/127.20.80.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
I20260929 10:50:14.243625 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.195:0
--local_ip_for_outbound_sockets=127.20.80.195
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:42107
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:43065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:14.323463 24217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:14.323675 24217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:14.323699 24217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:14.324846 24217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:14.324895 24217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:14.324946 24217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.195
I20260929 10:50:14.326431 24217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:43065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:42107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.195
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:14.326673 24217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:14.326902 24217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:14.331828 24222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:14.331845 24223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:14.331845 24225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:14.332191 24217 server_base.cc:1034] running on GCE node
I20260929 10:50:14.332422 24217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:14.332654 24217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:14.333803 24217 hybrid_clock.cc:648] HybridClock initialized: now 1790679014333796 us; error 35 us; skew 500 ppm
Sep 29 10:50:14 dist-test-slave-vqch krb5kdc[23839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679014, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:14.336184 24217 init.cc:377] Logged in from keytab as kudu/127.20.80.195@KRBTEST.COM (short username kudu)
I20260929 10:50:14.336603 24217 webserver.cc:460] Webserver started at http://127.20.80.195:46677/ using document root <none> and password file <none>
I20260929 10:50:14.336789 24217 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:14.336828 24217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:14.336922 24217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:14.337942 24217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-2/data/instance:
uuid: "2082d32ed8674459aef2175bb413905e"
format_stamp: "Formatted at 2026-09-29 10:50:14 on dist-test-slave-vqch"
server_key: "f1ed434a932d77b093025f4e1cc2eb35"
server_key_iv: "a6101f2fe1a0d45443a72ba24d72f436"
server_key_version: "encryptionkey@0"
I20260929 10:50:14.338238 24217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance:
uuid: "2082d32ed8674459aef2175bb413905e"
format_stamp: "Formatted at 2026-09-29 10:50:14 on dist-test-slave-vqch"
server_key: "f1ed434a932d77b093025f4e1cc2eb35"
server_key_iv: "a6101f2fe1a0d45443a72ba24d72f436"
server_key_version: "encryptionkey@0"
I20260929 10:50:14.339464 24217 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:50:14.340107 24232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:14.340276 24217 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:14.340363 24217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-2/wal
uuid: "2082d32ed8674459aef2175bb413905e"
format_stamp: "Formatted at 2026-09-29 10:50:14 on dist-test-slave-vqch"
server_key: "f1ed434a932d77b093025f4e1cc2eb35"
server_key_iv: "a6101f2fe1a0d45443a72ba24d72f436"
server_key_version: "encryptionkey@0"
I20260929 10:50:14.340433 24217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:14.352450 24217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:14.354223 24217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:14.354369 24217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:14.354573 24217 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:14.354872 24217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:14.354905 24217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:14.354944 24217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:14.354971 24217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:14.360601 24217 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.195:34441
I20260929 10:50:14.360646 24345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.195:34441 every 8 connection(s)
I20260929 10:50:14.360946 24217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
Sep 29 10:50:14 dist-test-slave-vqch krb5kdc[23839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679014, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:14.367900 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 24217
I20260929 10:50:14.368121 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance
I20260929 10:50:14.368330 20803 external_mini_cluster.cc:1428] Setting key dbc76960b9075d9ab928756436e8c11f
I20260929 10:50:14.370177 23927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:14.361643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:41725 (local address 127.20.80.254:42107)
0929 10:50:14.361708 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:14.361710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:14.361889 (+ 179us) server_negotiation.cc:407] Connection header received
0929 10:50:14.362346 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:14.362347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:14.362357 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:14.362403 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:14.363284 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:14.363771 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:14.364690 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:14.364884 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:14.364935 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:14.367140 (+ 2205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:14.367141 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:14.367142 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:14.367149 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:14.368771 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:14.369298 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:14.369299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:14.369300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:14.369376 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:14.369673 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:14.369674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:14.369675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:14.369790 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:14.369852 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:14.370056 (+ 204us) server_negotiation.cc:299] Negotiation successful
0929 10:50:14.370115 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 10:50:14.370361 24349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:14.361629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:42107 (local address 127.20.80.195:41725)
0929 10:50:14.361776 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:14.361810 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:14.362266 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:14.362464 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:14.362467 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:14.363015 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:14.363197 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:14.363202 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:14.363844 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:14.363845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:14.364622 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:14.364624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:14.364953 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:14.364954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:14.365115 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:14.365550 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:14.365558 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:14.367033 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:14.368866 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:14.368872 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:14.368874 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:14.369225 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:14.369446 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:14.369447 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:14.369447 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:14.369593 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:14.369850 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:14.369854 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:14.369963 (+ 109us) client_negotiation.cc:769] Sending connection context
0929 10:50:14.370042 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 10:50:14.370158 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
I20260929 10:50:14.370790 24346 heartbeater.cc:347] Connected to a master server at 127.20.80.254:42107
I20260929 10:50:14.370888 24346 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:14.371055 24346 heartbeater.cc:511] Master 127.20.80.254:42107 requested a full tablet report, sending...
I20260929 10:50:14.371424 23884 ts_manager.cc:194] Registered new tserver with Master: 2082d32ed8674459aef2175bb413905e (127.20.80.195:34441)
I20260929 10:50:14.371785 23884 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.195@KRBTEST.COM'} at 127.20.80.195:41725
I20260929 10:50:14.380554 20803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 10:50:14 dist-test-slave-vqch krb5kdc[23839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679014, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 29 10:50:14 dist-test-slave-vqch krb5kdc[23839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679014, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:14.403698 23927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:14.396336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53296 (local address 127.20.80.254:42107)
0929 10:50:14.396415 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:14.396418 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:14.396483 (+ 65us) server_negotiation.cc:407] Connection header received
0929 10:50:14.396578 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:14.396579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:14.396591 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:14.396658 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:14.397335 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:14.397944 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:14.398702 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:14.398895 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:14.398968 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:14.401171 (+ 2203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:14.401172 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:14.401173 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:14.401181 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:14.402551 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:14.402904 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:14.402905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:14.402905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:14.402960 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:14.403143 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:14.403144 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:14.403144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:14.403299 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:14.403407 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:14.403502 (+ 95us) server_negotiation.cc:299] Negotiation successful
0929 10:50:14.403586 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260929 10:50:14.403971 23884 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:53296
I20260929 10:50:14.404518 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 23946
I20260929 10:50:14.409214 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 24081
I20260929 10:50:14.413964 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 24217
I20260929 10:50:14.418251 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 23852
2026-09-29T10:50:14Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1856 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:50:14 dist-test-slave-vqch krb5kdc[24359](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:50:14 dist-test-slave-vqch krb5kdc[24359](info): set up 2 sockets
Sep 29 10:50:14 dist-test-slave-vqch krb5kdc[24359](info): commencing operation
krb5kdc: starting...
W20260929 10:50:15.456683 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:50:15 dist-test-slave-vqch krb5kdc[24359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:50:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:50:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.80.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
I20260929 10:50:15.569957 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:40815
--webserver_interface=127.20.80.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:41279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:40815
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:15.650324 24372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:15.650545 24372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:15.650581 24372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:15.651716 24372 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:50:15.651780 24372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:15.651805 24372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:15.651839 24372 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:50:15.651861 24372 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:50:15.653398 24372 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:41279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:40815
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:40815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:15.653656 24372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:15.653874 24372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:15.658542 24378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:15.658553 24380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:15.658704 24372 server_base.cc:1034] running on GCE node
W20260929 10:50:15.658567 24377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:15.658986 24372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:15.659260 24372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:15.660441 24372 hybrid_clock.cc:648] HybridClock initialized: now 1790679015660432 us; error 33 us; skew 500 ppm
Sep 29 10:50:15 dist-test-slave-vqch krb5kdc[24359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679015, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:15.662917 24372 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:50:15.663327 24372 webserver.cc:460] Webserver started at http://127.20.80.254:42761/ using document root <none> and password file <none>
I20260929 10:50:15.663569 24372 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:15.663638 24372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:15.663774 24372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:15.664795 24372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "5ac973d83ea84549820b5ae7af59e5db"
format_stamp: "Formatted at 2026-09-29 10:50:15 on dist-test-slave-vqch"
server_key: "adb53613b473d79f7c7d9cfc17712d80"
server_key_iv: "e0fdd401ca41fabfe8e5ba008584d30e"
server_key_version: "encryptionkey@0"
I20260929 10:50:15.665177 24372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "5ac973d83ea84549820b5ae7af59e5db"
format_stamp: "Formatted at 2026-09-29 10:50:15 on dist-test-slave-vqch"
server_key: "adb53613b473d79f7c7d9cfc17712d80"
server_key_iv: "e0fdd401ca41fabfe8e5ba008584d30e"
server_key_version: "encryptionkey@0"
I20260929 10:50:15.666457 24372 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:50:15.667136 24387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:15.667306 24372 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:50:15.667407 24372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "5ac973d83ea84549820b5ae7af59e5db"
format_stamp: "Formatted at 2026-09-29 10:50:15 on dist-test-slave-vqch"
server_key: "adb53613b473d79f7c7d9cfc17712d80"
server_key_iv: "e0fdd401ca41fabfe8e5ba008584d30e"
server_key_version: "encryptionkey@0"
I20260929 10:50:15.667488 24372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:15.683563 24372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:15.685883 24372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:15.686045 24372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:15.689960 24372 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:40815
I20260929 10:50:15.689997 24439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:40815 every 8 connection(s)
I20260929 10:50:15.690526 24372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:50:15.691219 24440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:15.693594 24440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ac973d83ea84549820b5ae7af59e5db: Bootstrap starting.
I20260929 10:50:15.694185 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 24372
I20260929 10:50:15.694295 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:50:15.694317 24440 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ac973d83ea84549820b5ae7af59e5db: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:15.694518 20803 external_mini_cluster.cc:1428] Setting key 879f1c399e59fdb55657b6d63d5b07aa
I20260929 10:50:15.694648 24440 log.cc:824] T 00000000000000000000000000000000 P 5ac973d83ea84549820b5ae7af59e5db: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:15.695452 24440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ac973d83ea84549820b5ae7af59e5db: No bootstrap required, opened a new log
I20260929 10:50:15.697911 24440 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ac973d83ea84549820b5ae7af59e5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ac973d83ea84549820b5ae7af59e5db" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 40815 } }
I20260929 10:50:15.698081 24440 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ac973d83ea84549820b5ae7af59e5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:15.698117 24440 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ac973d83ea84549820b5ae7af59e5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ac973d83ea84549820b5ae7af59e5db, State: Initialized, Role: FOLLOWER
I20260929 10:50:15.698228 24440 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ac973d83ea84549820b5ae7af59e5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ac973d83ea84549820b5ae7af59e5db" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 40815 } }
I20260929 10:50:15.698295 24440 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ac973d83ea84549820b5ae7af59e5db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:50:15.698323 24440 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ac973d83ea84549820b5ae7af59e5db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:50:15.698374 24440 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ac973d83ea84549820b5ae7af59e5db [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:15.698982 24440 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ac973d83ea84549820b5ae7af59e5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ac973d83ea84549820b5ae7af59e5db" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 40815 } }
I20260929 10:50:15.699070 24440 leader_election.cc:302] T 00000000000000000000000000000000 P 5ac973d83ea84549820b5ae7af59e5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ac973d83ea84549820b5ae7af59e5db; no voters:
I20260929 10:50:15.699311 24440 leader_election.cc:288] T 00000000000000000000000000000000 P 5ac973d83ea84549820b5ae7af59e5db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:50:15.699337 24445 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ac973d83ea84549820b5ae7af59e5db [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:15.699512 24445 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ac973d83ea84549820b5ae7af59e5db [term 1 LEADER]: Becoming Leader. State: Replica: 5ac973d83ea84549820b5ae7af59e5db, State: Running, Role: LEADER
I20260929 10:50:15.699620 24440 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ac973d83ea84549820b5ae7af59e5db [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:50:15.699662 24445 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ac973d83ea84549820b5ae7af59e5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ac973d83ea84549820b5ae7af59e5db" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 40815 } }
I20260929 10:50:15.700213 24446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ac973d83ea84549820b5ae7af59e5db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ac973d83ea84549820b5ae7af59e5db. Latest consensus state: current_term: 1 leader_uuid: "5ac973d83ea84549820b5ae7af59e5db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ac973d83ea84549820b5ae7af59e5db" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 40815 } } }
I20260929 10:50:15.700304 24446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ac973d83ea84549820b5ae7af59e5db [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:15.700915 24454 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:50:15.701380 24454 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 29 10:50:15 dist-test-slave-vqch krb5kdc[24359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:15.701769 24447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ac973d83ea84549820b5ae7af59e5db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ac973d83ea84549820b5ae7af59e5db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ac973d83ea84549820b5ae7af59e5db" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 40815 } } }
I20260929 10:50:15.701850 24447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ac973d83ea84549820b5ae7af59e5db [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:15.702927 24454 catalog_manager.cc:1347] Generated new cluster ID: c5131462f3984a06bf5d238367cac641
I20260929 10:50:15.702975 24454 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:50:15.705920 24444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:15.695071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55936 (local address 127.20.80.254:40815)
0929 10:50:15.695206 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:15.695215 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:15.695248 (+ 33us) server_negotiation.cc:407] Connection header received
0929 10:50:15.695726 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:15.695733 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:15.695781 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:15.695901 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:15.696673 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:15.697569 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:15.698630 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:15.699029 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:15.699157 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:15.702032 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:15.702036 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:15.702036 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:15.702047 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:15.704275 (+ 2228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:15.704649 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:15.704652 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:15.704653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:15.704763 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:15.704966 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:15.704967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:15.704967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:15.705302 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:15.705458 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:15.705561 (+ 103us) server_negotiation.cc:299] Negotiation successful
0929 10:50:15.705699 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":67,"threads_started":1}
I20260929 10:50:15.714411 24454 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:50:15.714915 24454 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:50:15.719992 24454 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ac973d83ea84549820b5ae7af59e5db: Generated new TSK 0
I20260929 10:50:15.720172 24454 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.80.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
I20260929 10:50:15.748216 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.193:0
--local_ip_for_outbound_sockets=127.20.80.193
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:40815
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:41279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:15.829381 24466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:15.829574 24466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:15.829597 24466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:15.830659 24466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:15.830693 24466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:15.830717 24466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.193
I20260929 10:50:15.832095 24466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:41279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:40815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.193
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:15.832309 24466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:15.832530 24466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:15.837656 24474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:15.837790 24466 server_base.cc:1034] running on GCE node
W20260929 10:50:15.837682 24471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:15.837883 24472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:15.838091 24466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:15.838300 24466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:15.839444 24466 hybrid_clock.cc:648] HybridClock initialized: now 1790679015839434 us; error 39 us; skew 500 ppm
Sep 29 10:50:15 dist-test-slave-vqch krb5kdc[24359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679015, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:15.841976 24466 init.cc:377] Logged in from keytab as kudu/127.20.80.193@KRBTEST.COM (short username kudu)
I20260929 10:50:15.842360 24466 webserver.cc:460] Webserver started at http://127.20.80.193:38761/ using document root <none> and password file <none>
I20260929 10:50:15.842587 24466 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:15.842633 24466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:15.842720 24466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:15.843706 24466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-0/data/instance:
uuid: "126964d257824919a3f77c788f1aaefe"
format_stamp: "Formatted at 2026-09-29 10:50:15 on dist-test-slave-vqch"
server_key: "940f9cdff96536e01f7f7ddad9008954"
server_key_iv: "69431128e856db110ea4fdbd07e297a4"
server_key_version: "encryptionkey@0"
I20260929 10:50:15.844004 24466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance:
uuid: "126964d257824919a3f77c788f1aaefe"
format_stamp: "Formatted at 2026-09-29 10:50:15 on dist-test-slave-vqch"
server_key: "940f9cdff96536e01f7f7ddad9008954"
server_key_iv: "69431128e856db110ea4fdbd07e297a4"
server_key_version: "encryptionkey@0"
I20260929 10:50:15.845315 24466 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:50:15.845995 24481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:15.846185 24466 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:15.846272 24466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-0/wal
uuid: "126964d257824919a3f77c788f1aaefe"
format_stamp: "Formatted at 2026-09-29 10:50:15 on dist-test-slave-vqch"
server_key: "940f9cdff96536e01f7f7ddad9008954"
server_key_iv: "69431128e856db110ea4fdbd07e297a4"
server_key_version: "encryptionkey@0"
I20260929 10:50:15.846354 24466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:15.857242 24466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:15.859062 24466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:15.859200 24466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:15.859436 24466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:15.859731 24466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:15.859764 24466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:15.859807 24466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:15.859833 24466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:15.865633 24466 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.193:35485
I20260929 10:50:15.865700 24595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.193:35485 every 8 connection(s)
I20260929 10:50:15.866017 24466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
Sep 29 10:50:15 dist-test-slave-vqch krb5kdc[24359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679015, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:15.873103 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 24466
I20260929 10:50:15.873210 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance
I20260929 10:50:15.873420 20803 external_mini_cluster.cc:1428] Setting key be25b6f5d34f1cca355557f0f32aa37e
I20260929 10:50:15.875806 24444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:15.866777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:58425 (local address 127.20.80.254:40815)
0929 10:50:15.866835 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:15.866837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:15.867045 (+ 208us) server_negotiation.cc:407] Connection header received
0929 10:50:15.867916 (+ 871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:15.867918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:15.867928 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:15.867989 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:15.868560 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:15.869147 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:15.870135 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:15.870370 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:15.870449 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:15.872622 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:15.872624 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:15.872624 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:15.872633 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:15.874437 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:15.874831 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:15.874833 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:15.874833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:15.874904 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:15.875154 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:15.875155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:15.875155 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:15.875272 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:15.875346 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:15.875627 (+ 281us) server_negotiation.cc:299] Negotiation successful
0929 10:50:15.875715 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260929 10:50:15.876030 24598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:15.866740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:40815 (local address 127.20.80.193:58425)
0929 10:50:15.866896 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:15.866941 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:15.867819 (+ 878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:15.868042 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:15.868045 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:15.868288 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:15.868475 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:15.868479 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:15.869240 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:15.869240 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:15.870042 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:15.870044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:15.870457 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:15.870459 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:15.870578 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:15.871146 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:15.871188 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:15.872522 (+ 1334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:15.874525 (+ 2003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:15.874532 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:15.874534 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:15.874753 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:15.874968 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:15.874969 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:15.874969 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:15.875081 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:15.875348 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:15.875356 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:15.875498 (+ 142us) client_negotiation.cc:769] Sending connection context
0929 10:50:15.875606 (+ 108us) client_negotiation.cc:240] Negotiation successful
0929 10:50:15.875757 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
I20260929 10:50:15.876473 24596 heartbeater.cc:347] Connected to a master server at 127.20.80.254:40815
I20260929 10:50:15.876572 24596 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:15.876757 24596 heartbeater.cc:511] Master 127.20.80.254:40815 requested a full tablet report, sending...
I20260929 10:50:15.877331 24404 ts_manager.cc:194] Registered new tserver with Master: 126964d257824919a3f77c788f1aaefe (127.20.80.193:35485)
I20260929 10:50:15.877950 24404 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.193@KRBTEST.COM'} at 127.20.80.193:58425
WARNING: no policy specified for kudu/127.20.80.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
I20260929 10:50:15.899289 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.194:0
--local_ip_for_outbound_sockets=127.20.80.194
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:40815
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:41279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:15.986068 24601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:15.986286 24601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:15.986312 24601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:15.987414 24601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:15.987462 24601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:15.987490 24601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.194
I20260929 10:50:15.988973 24601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:41279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:40815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.194
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:15.989192 24601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:15.989374 24601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:15.994189 24609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:15.994199 24607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:15.994213 24606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:15.994379 24601 server_base.cc:1034] running on GCE node
I20260929 10:50:15.994573 24601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:15.994804 24601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:15.995980 24601 hybrid_clock.cc:648] HybridClock initialized: now 1790679015995967 us; error 30 us; skew 500 ppm
Sep 29 10:50:15 dist-test-slave-vqch krb5kdc[24359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679015, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:15.998373 24601 init.cc:377] Logged in from keytab as kudu/127.20.80.194@KRBTEST.COM (short username kudu)
I20260929 10:50:15.998735 24601 webserver.cc:460] Webserver started at http://127.20.80.194:35857/ using document root <none> and password file <none>
I20260929 10:50:15.998924 24601 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:15.998963 24601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:15.999049 24601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:16.000108 24601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-1/data/instance:
uuid: "398e96fd32194291be47f0091762b529"
format_stamp: "Formatted at 2026-09-29 10:50:15 on dist-test-slave-vqch"
server_key: "2df2a205329ab48454001b9b5c73bdf1"
server_key_iv: "5416ce74d64ca29f175fca8053a5a5e3"
server_key_version: "encryptionkey@0"
I20260929 10:50:16.000403 24601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance:
uuid: "398e96fd32194291be47f0091762b529"
format_stamp: "Formatted at 2026-09-29 10:50:15 on dist-test-slave-vqch"
server_key: "2df2a205329ab48454001b9b5c73bdf1"
server_key_iv: "5416ce74d64ca29f175fca8053a5a5e3"
server_key_version: "encryptionkey@0"
I20260929 10:50:16.001840 24601 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:50:16.002552 24616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:16.002725 24601 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:16.002818 24601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-1/wal
uuid: "398e96fd32194291be47f0091762b529"
format_stamp: "Formatted at 2026-09-29 10:50:15 on dist-test-slave-vqch"
server_key: "2df2a205329ab48454001b9b5c73bdf1"
server_key_iv: "5416ce74d64ca29f175fca8053a5a5e3"
server_key_version: "encryptionkey@0"
I20260929 10:50:16.002912 24601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:16.022792 24601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:16.024688 24601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:16.024843 24601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:16.025085 24601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:16.025372 24601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:16.025429 24601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:16.025466 24601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:16.025494 24601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:16.031104 24601 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.194:35225
I20260929 10:50:16.031177 24730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.194:35225 every 8 connection(s)
I20260929 10:50:16.031478 24601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
I20260929 10:50:16.034307 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 24601
I20260929 10:50:16.034396 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance
I20260929 10:50:16.034631 20803 external_mini_cluster.cc:1428] Setting key 07d8882f18b09eae7e2a31b1765997db
Sep 29 10:50:16 dist-test-slave-vqch krb5kdc[24359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679015, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:16.042332 24444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:16.032197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:54207 (local address 127.20.80.254:40815)
0929 10:50:16.032274 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:16.032277 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:16.032424 (+ 147us) server_negotiation.cc:407] Connection header received
0929 10:50:16.033330 (+ 906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:16.033332 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:16.033342 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:16.033402 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:16.033964 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:16.034560 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:16.036107 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:16.036513 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:16.036587 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:16.039412 (+ 2825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:16.039413 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:16.039414 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:16.039421 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:16.040845 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:16.041390 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:16.041391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:16.041391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:16.041443 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:16.041725 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:16.041725 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:16.041726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:16.041823 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:16.041884 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:16.042204 (+ 320us) server_negotiation.cc:299] Negotiation successful
0929 10:50:16.042267 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 10:50:16.042641 24733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:16.032148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:40815 (local address 127.20.80.194:54207)
0929 10:50:16.032300 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:16.032332 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:16.033245 (+ 913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:16.033457 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:16.033460 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:16.033697 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:16.033882 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:16.033886 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:16.034643 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:16.034644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:16.035993 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:16.035996 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:16.036662 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:16.036664 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:16.036815 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:16.037389 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:16.037399 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:16.039294 (+ 1895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:16.040945 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:16.040953 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:16.040955 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:16.041301 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:16.041510 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:16.041510 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:16.041511 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:16.041657 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:16.041887 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:16.041893 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:16.042058 (+ 165us) client_negotiation.cc:769] Sending connection context
0929 10:50:16.042174 (+ 116us) client_negotiation.cc:240] Negotiation successful
0929 10:50:16.042358 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":91,"threads_started":1}
I20260929 10:50:16.043152 24731 heartbeater.cc:347] Connected to a master server at 127.20.80.254:40815
I20260929 10:50:16.043258 24731 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:16.043470 24731 heartbeater.cc:511] Master 127.20.80.254:40815 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.20.80.195@KRBTEST.COM; defaulting to no policy
I20260929 10:50:16.043850 24404 ts_manager.cc:194] Registered new tserver with Master: 398e96fd32194291be47f0091762b529 (127.20.80.194:35225)
I20260929 10:50:16.044322 24404 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.194@KRBTEST.COM'} at 127.20.80.194:54207
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
I20260929 10:50:16.061723 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.195:0
--local_ip_for_outbound_sockets=127.20.80.195
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:40815
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:41279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:16.141415 24736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:16.141670 24736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:16.141708 24736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:16.142899 24736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:16.142973 24736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:16.143018 24736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.195
I20260929 10:50:16.144572 24736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:41279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:40815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.195
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:16.144861 24736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:16.145180 24736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:16.150285 24742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:16.150246 24741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:16.150385 24736 server_base.cc:1034] running on GCE node
W20260929 10:50:16.150250 24744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:16.150713 24736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:16.150913 24736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:16.152057 24736 hybrid_clock.cc:648] HybridClock initialized: now 1790679016152045 us; error 35 us; skew 500 ppm
Sep 29 10:50:16 dist-test-slave-vqch krb5kdc[24359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679016, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:16.154605 24736 init.cc:377] Logged in from keytab as kudu/127.20.80.195@KRBTEST.COM (short username kudu)
I20260929 10:50:16.155099 24736 webserver.cc:460] Webserver started at http://127.20.80.195:37555/ using document root <none> and password file <none>
I20260929 10:50:16.155328 24736 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:16.155395 24736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:16.155542 24736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:16.156662 24736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-2/data/instance:
uuid: "e60b6425d4ce4d00b35f80dda70b2310"
format_stamp: "Formatted at 2026-09-29 10:50:16 on dist-test-slave-vqch"
server_key: "0d873e447a91864e9f5647001814971e"
server_key_iv: "12603969640cad7ec59da1cb9003cc45"
server_key_version: "encryptionkey@0"
I20260929 10:50:16.157097 24736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance:
uuid: "e60b6425d4ce4d00b35f80dda70b2310"
format_stamp: "Formatted at 2026-09-29 10:50:16 on dist-test-slave-vqch"
server_key: "0d873e447a91864e9f5647001814971e"
server_key_iv: "12603969640cad7ec59da1cb9003cc45"
server_key_version: "encryptionkey@0"
I20260929 10:50:16.158390 24736 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:16.159108 24751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:16.159271 24736 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:16.159369 24736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-2/wal
uuid: "e60b6425d4ce4d00b35f80dda70b2310"
format_stamp: "Formatted at 2026-09-29 10:50:16 on dist-test-slave-vqch"
server_key: "0d873e447a91864e9f5647001814971e"
server_key_iv: "12603969640cad7ec59da1cb9003cc45"
server_key_version: "encryptionkey@0"
I20260929 10:50:16.159451 24736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:16.173213 24736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:16.175132 24736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:16.175292 24736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:16.175514 24736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:16.175839 24736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:16.175894 24736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:16.175926 24736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:16.175954 24736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:16.181720 24736 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.195:39753
I20260929 10:50:16.181768 24864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.195:39753 every 8 connection(s)
I20260929 10:50:16.182066 24736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
I20260929 10:50:16.186568 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 24736
I20260929 10:50:16.186691 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance
I20260929 10:50:16.186909 20803 external_mini_cluster.cc:1428] Setting key 27ad146e50bbac64b57c6d2a323ebd34
Sep 29 10:50:16 dist-test-slave-vqch krb5kdc[24359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679016, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:16.192133 24444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:16.182777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:56613 (local address 127.20.80.254:40815)
0929 10:50:16.182857 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:16.182861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:16.183043 (+ 182us) server_negotiation.cc:407] Connection header received
0929 10:50:16.184079 (+ 1036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:16.184081 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:16.184093 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:16.184175 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:16.184836 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:16.185474 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:16.186668 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:16.187022 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:16.187098 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:16.189495 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:16.189496 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:16.189497 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:16.189504 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:16.190925 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:16.191308 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:16.191309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:16.191309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:16.191379 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:16.191607 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:16.191608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:16.191608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:16.191715 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:16.191768 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:16.192000 (+ 232us) server_negotiation.cc:299] Negotiation successful
0929 10:50:16.192069 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260929 10:50:16.192270 24867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:16.182721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:40815 (local address 127.20.80.195:56613)
0929 10:50:16.182916 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:16.182960 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:16.183959 (+ 999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:16.184241 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:16.184245 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:16.184526 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:16.184726 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:16.184730 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:16.185573 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:16.185575 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:16.186513 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:16.186515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:16.187119 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:16.187120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:16.187284 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:16.187872 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:16.187882 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:16.189410 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:16.191012 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:16.191017 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:16.191019 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:16.191221 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:16.191447 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:16.191448 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:16.191448 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:16.191550 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:16.191772 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:16.191775 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:16.191890 (+ 115us) client_negotiation.cc:769] Sending connection context
0929 10:50:16.191972 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 10:50:16.192088 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":122,"threads_started":1}
I20260929 10:50:16.192727 24865 heartbeater.cc:347] Connected to a master server at 127.20.80.254:40815
I20260929 10:50:16.192837 24865 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:16.193081 24865 heartbeater.cc:511] Master 127.20.80.254:40815 requested a full tablet report, sending...
I20260929 10:50:16.193491 24404 ts_manager.cc:194] Registered new tserver with Master: e60b6425d4ce4d00b35f80dda70b2310 (127.20.80.195:39753)
I20260929 10:50:16.193851 24404 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.195@KRBTEST.COM'} at 127.20.80.195:56613
I20260929 10:50:16.198717 20803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 10:50:16 dist-test-slave-vqch krb5kdc[24359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 10:50:16 dist-test-slave-vqch krb5kdc[24359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.193@KRBTEST.COM
I20260929 10:50:16.221813 24872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:16.213921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33028 (local address 127.20.80.193:35485)
0929 10:50:16.214114 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:16.214119 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:16.214129 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:50:16.214154 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:16.214156 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:16.214170 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:16.214236 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:16.214788 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:16.215543 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:16.216271 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:16.216604 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:16.216661 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:16.219161 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:16.219169 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:16.219169 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:16.219175 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:16.220782 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:16.221136 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:16.221139 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:16.221139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:16.221203 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:16.221350 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:16.221351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:16.221351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:16.221470 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:16.221566 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:16.221635 (+ 69us) server_negotiation.cc:299] Negotiation successful
0929 10:50:16.221694 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":114,"threads_started":1}
Sep 29 10:50:16 dist-test-slave-vqch krb5kdc[24359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:16.229682 24444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:16.223090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55950 (local address 127.20.80.254:40815)
0929 10:50:16.223139 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:16.223140 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:16.223148 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:50:16.223189 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:16.223190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:16.223199 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:16.223243 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:16.223659 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:16.224145 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:16.224773 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:16.224966 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:16.225093 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:16.227172 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:16.227174 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:16.227174 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:16.227181 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:16.228523 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:16.228838 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:16.228839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:16.228839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:16.228910 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:16.229137 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:16.229138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:16.229138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:16.229282 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:16.229369 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:16.229550 (+ 181us) server_negotiation.cc:299] Negotiation successful
0929 10:50:16.229615 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260929 10:50:16.229947 24404 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55950
I20260929 10:50:16.230337 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 24466
I20260929 10:50:16.235313 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 24601
I20260929 10:50:16.239663 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 24736
I20260929 10:50:16.243961 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 24372
2026-09-29T10:50:16Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1825 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:50:16 dist-test-slave-vqch krb5kdc[24877](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:50:16 dist-test-slave-vqch krb5kdc[24877](info): set up 2 sockets
Sep 29 10:50:16 dist-test-slave-vqch krb5kdc[24877](info): commencing operation
krb5kdc: starting...
W20260929 10:50:17.281106 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:50:17 dist-test-slave-vqch krb5kdc[24877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:50:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:50:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.80.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
I20260929 10:50:17.400107 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:34291
--webserver_interface=127.20.80.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:41219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:34291
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:17.479472 24891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:17.479712 24891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:17.479753 24891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:17.480891 24891 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:50:17.480952 24891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:17.480976 24891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:17.481022 24891 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:50:17.481050 24891 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:50:17.482776 24891 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:41219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:34291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:34291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:17.483078 24891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:17.483304 24891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:17.488107 24899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:17.488133 24891 server_base.cc:1034] running on GCE node
W20260929 10:50:17.488103 24897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:17.488103 24896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:17.488583 24891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:17.488895 24891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:17.490056 24891 hybrid_clock.cc:648] HybridClock initialized: now 1790679017490060 us; error 44 us; skew 500 ppm
Sep 29 10:50:17 dist-test-slave-vqch krb5kdc[24877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679017, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:17.492509 24891 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:50:17.492899 24891 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:17.492967 24891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:17.493136 24891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:17.494189 24891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "f473f2db0e09439d8c96972c5357949a"
format_stamp: "Formatted at 2026-09-29 10:50:17 on dist-test-slave-vqch"
server_key: "c0d058f623d3ed5a63862ee588511ff9"
server_key_iv: "d4a9382057522bef5bbdb254e33e997f"
server_key_version: "encryptionkey@0"
I20260929 10:50:17.494539 24891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "f473f2db0e09439d8c96972c5357949a"
format_stamp: "Formatted at 2026-09-29 10:50:17 on dist-test-slave-vqch"
server_key: "c0d058f623d3ed5a63862ee588511ff9"
server_key_iv: "d4a9382057522bef5bbdb254e33e997f"
server_key_version: "encryptionkey@0"
I20260929 10:50:17.495855 24891 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:50:17.496601 24905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:17.496825 24891 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:17.496928 24891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "f473f2db0e09439d8c96972c5357949a"
format_stamp: "Formatted at 2026-09-29 10:50:17 on dist-test-slave-vqch"
server_key: "c0d058f623d3ed5a63862ee588511ff9"
server_key_iv: "d4a9382057522bef5bbdb254e33e997f"
server_key_version: "encryptionkey@0"
I20260929 10:50:17.497042 24891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:17.513453 24891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:17.515631 24891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:17.515787 24891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:17.519794 24891 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:34291
I20260929 10:50:17.519833 24958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:34291 every 8 connection(s)
I20260929 10:50:17.520125 24891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:50:17.521111 24959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:17.523418 24959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f473f2db0e09439d8c96972c5357949a: Bootstrap starting.
I20260929 10:50:17.524029 24959 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f473f2db0e09439d8c96972c5357949a: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:17.524336 24959 log.cc:824] T 00000000000000000000000000000000 P f473f2db0e09439d8c96972c5357949a: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:17.524883 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 24891
I20260929 10:50:17.524967 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:50:17.525168 20803 external_mini_cluster.cc:1428] Setting key eafa72dc09f9c77049ac04cfa27b35d3
I20260929 10:50:17.525159 24959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f473f2db0e09439d8c96972c5357949a: No bootstrap required, opened a new log
I20260929 10:50:17.528108 24959 raft_consensus.cc:348] T 00000000000000000000000000000000 P f473f2db0e09439d8c96972c5357949a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f473f2db0e09439d8c96972c5357949a" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 34291 } }
I20260929 10:50:17.528241 24959 raft_consensus.cc:374] T 00000000000000000000000000000000 P f473f2db0e09439d8c96972c5357949a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:17.528275 24959 raft_consensus.cc:729] T 00000000000000000000000000000000 P f473f2db0e09439d8c96972c5357949a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f473f2db0e09439d8c96972c5357949a, State: Initialized, Role: FOLLOWER
I20260929 10:50:17.528420 24959 consensus_queue.cc:260] T 00000000000000000000000000000000 P f473f2db0e09439d8c96972c5357949a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f473f2db0e09439d8c96972c5357949a" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 34291 } }
I20260929 10:50:17.528491 24959 raft_consensus.cc:388] T 00000000000000000000000000000000 P f473f2db0e09439d8c96972c5357949a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:50:17.528512 24959 raft_consensus.cc:482] T 00000000000000000000000000000000 P f473f2db0e09439d8c96972c5357949a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:50:17.528581 24959 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f473f2db0e09439d8c96972c5357949a [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:17.529287 24959 raft_consensus.cc:504] T 00000000000000000000000000000000 P f473f2db0e09439d8c96972c5357949a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f473f2db0e09439d8c96972c5357949a" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 34291 } }
I20260929 10:50:17.529402 24959 leader_election.cc:302] T 00000000000000000000000000000000 P f473f2db0e09439d8c96972c5357949a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f473f2db0e09439d8c96972c5357949a; no voters:
I20260929 10:50:17.529554 24959 leader_election.cc:288] T 00000000000000000000000000000000 P f473f2db0e09439d8c96972c5357949a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:50:17.529610 24964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f473f2db0e09439d8c96972c5357949a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:17.529770 24964 raft_consensus.cc:686] T 00000000000000000000000000000000 P f473f2db0e09439d8c96972c5357949a [term 1 LEADER]: Becoming Leader. State: Replica: f473f2db0e09439d8c96972c5357949a, State: Running, Role: LEADER
I20260929 10:50:17.529819 24959 sys_catalog.cc:564] T 00000000000000000000000000000000 P f473f2db0e09439d8c96972c5357949a [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:50:17.529896 24964 consensus_queue.cc:237] T 00000000000000000000000000000000 P f473f2db0e09439d8c96972c5357949a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f473f2db0e09439d8c96972c5357949a" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 34291 } }
I20260929 10:50:17.530241 24964 sys_catalog.cc:455] T 00000000000000000000000000000000 P f473f2db0e09439d8c96972c5357949a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f473f2db0e09439d8c96972c5357949a. Latest consensus state: current_term: 1 leader_uuid: "f473f2db0e09439d8c96972c5357949a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f473f2db0e09439d8c96972c5357949a" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 34291 } } }
I20260929 10:50:17.530251 24965 sys_catalog.cc:455] T 00000000000000000000000000000000 P f473f2db0e09439d8c96972c5357949a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f473f2db0e09439d8c96972c5357949a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f473f2db0e09439d8c96972c5357949a" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 34291 } } }
I20260929 10:50:17.530411 24965 sys_catalog.cc:458] T 00000000000000000000000000000000 P f473f2db0e09439d8c96972c5357949a [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:17.530624 24964 sys_catalog.cc:458] T 00000000000000000000000000000000 P f473f2db0e09439d8c96972c5357949a [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:17.530776 24973 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:50:17.531312 24973 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:50:17.532938 24973 catalog_manager.cc:1347] Generated new cluster ID: f04b4976184e4d659ab532840a621896
I20260929 10:50:17.533008 24973 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 10:50:17 dist-test-slave-vqch krb5kdc[24877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:17.537086 24963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:17.525749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57566 (local address 127.20.80.254:34291)
0929 10:50:17.525944 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:17.525952 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:17.525978 (+ 26us) server_negotiation.cc:407] Connection header received
0929 10:50:17.526572 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:17.526579 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:17.526625 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:17.526732 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:17.527549 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:17.528511 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:17.529666 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:17.530304 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:17.530435 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:17.533743 (+ 3308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:17.533747 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:17.533747 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:17.533764 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:17.535393 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:17.535779 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:17.535782 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:17.535782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:17.535869 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:17.536077 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:17.536078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:17.536078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:17.536310 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:17.536482 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:17.536550 (+ 68us) server_negotiation.cc:299] Negotiation successful
0929 10:50:17.536686 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":74,"threads_started":1}
I20260929 10:50:17.551252 24973 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:50:17.551791 24973 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:50:17.564026 24973 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f473f2db0e09439d8c96972c5357949a: Generated new TSK 0
I20260929 10:50:17.564218 24973 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.80.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
I20260929 10:50:17.596306 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.193:0
--local_ip_for_outbound_sockets=127.20.80.193
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:34291
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:41219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:17.679858 24985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:17.680091 24985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:17.680132 24985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:17.681375 24985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:17.681435 24985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:17.681484 24985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.193
I20260929 10:50:17.683000 24985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:41219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:34291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.193
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:17.683256 24985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:17.683483 24985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:17.688616 24991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:17.688625 24994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:17.688866 24985 server_base.cc:1034] running on GCE node
W20260929 10:50:17.688638 24992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:17.689183 24985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:17.689426 24985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:17.690606 24985 hybrid_clock.cc:648] HybridClock initialized: now 1790679017690592 us; error 43 us; skew 500 ppm
Sep 29 10:50:17 dist-test-slave-vqch krb5kdc[24877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679017, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:17.693243 24985 init.cc:377] Logged in from keytab as kudu/127.20.80.193@KRBTEST.COM (short username kudu)
I20260929 10:50:17.693589 24985 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:17.693637 24985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:17.693727 24985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:17.694707 24985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-0/data/instance:
uuid: "c81986b3f6f54360bc43bfbf4470ac7e"
format_stamp: "Formatted at 2026-09-29 10:50:17 on dist-test-slave-vqch"
server_key: "16aa47d437a7f2414b719c9dcf65752f"
server_key_iv: "efba9d58f44f390202591f73e17c18be"
server_key_version: "encryptionkey@0"
I20260929 10:50:17.695001 24985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance:
uuid: "c81986b3f6f54360bc43bfbf4470ac7e"
format_stamp: "Formatted at 2026-09-29 10:50:17 on dist-test-slave-vqch"
server_key: "16aa47d437a7f2414b719c9dcf65752f"
server_key_iv: "efba9d58f44f390202591f73e17c18be"
server_key_version: "encryptionkey@0"
I20260929 10:50:17.696233 24985 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 10:50:17.696897 25000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:17.697062 24985 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:17.697144 24985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-0/wal
uuid: "c81986b3f6f54360bc43bfbf4470ac7e"
format_stamp: "Formatted at 2026-09-29 10:50:17 on dist-test-slave-vqch"
server_key: "16aa47d437a7f2414b719c9dcf65752f"
server_key_iv: "efba9d58f44f390202591f73e17c18be"
server_key_version: "encryptionkey@0"
I20260929 10:50:17.697197 24985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:17.712055 24985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:17.713989 24985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:17.714095 24985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:17.714269 24985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:17.714573 24985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:17.714605 24985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:17.714627 24985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:17.714672 24985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:17.720101 24985 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.193:35729
I20260929 10:50:17.720153 25114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.193:35729 every 8 connection(s)
I20260929 10:50:17.720423 24985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
I20260929 10:50:17.721542 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 24985
I20260929 10:50:17.721629 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance
I20260929 10:50:17.721817 20803 external_mini_cluster.cc:1428] Setting key 3c806dfe1d8dd86b615bb6b7e54f5f05
Sep 29 10:50:17 dist-test-slave-vqch krb5kdc[24877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679017, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.80.194@KRBTEST.COM; defaulting to no policy
I20260929 10:50:17.730823 24963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:17.721184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:37989 (local address 127.20.80.254:34291)
0929 10:50:17.721227 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:17.721229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:17.721438 (+ 209us) server_negotiation.cc:407] Connection header received
0929 10:50:17.722096 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:17.722098 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:17.722112 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:17.722185 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:17.723343 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:17.724116 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:17.725179 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:17.725415 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:17.725499 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:17.727855 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:17.727856 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:17.727857 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:17.727865 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:17.729296 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:17.729771 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:17.729773 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:17.729773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:17.729837 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:17.730112 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:17.730113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:17.730113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:17.730226 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:17.730302 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:17.730596 (+ 294us) server_negotiation.cc:299] Negotiation successful
0929 10:50:17.730721 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 10:50:17.730954 25117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:17.721144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:34291 (local address 127.20.80.193:37989)
0929 10:50:17.721309 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:17.721350 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:17.721949 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:17.722272 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:17.722276 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:17.722629 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:17.723131 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:17.723143 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:17.724216 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:17.724217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:17.725076 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:17.725079 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:17.725510 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:17.725511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:17.725642 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:17.726158 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:17.726166 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:17.727755 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:17.729402 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:17.729409 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:17.729412 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:17.729694 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:17.729904 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:17.729905 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:17.729906 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:17.730031 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:17.730292 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:17.730299 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:17.730453 (+ 154us) client_negotiation.cc:769] Sending connection context
0929 10:50:17.730562 (+ 109us) client_negotiation.cc:240] Negotiation successful
0929 10:50:17.730722 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
I20260929 10:50:17.731467 25115 heartbeater.cc:347] Connected to a master server at 127.20.80.254:34291
I20260929 10:50:17.731592 25115 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:17.731731 25115 heartbeater.cc:511] Master 127.20.80.254:34291 requested a full tablet report, sending...
I20260929 10:50:17.732242 24923 ts_manager.cc:194] Registered new tserver with Master: c81986b3f6f54360bc43bfbf4470ac7e (127.20.80.193:35729)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.194@KRBTEST.COM" created.
I20260929 10:50:17.733165 24923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.193@KRBTEST.COM'} at 127.20.80.193:37989
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
I20260929 10:50:17.747788 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.194:0
--local_ip_for_outbound_sockets=127.20.80.194
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:34291
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:41219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:17.826725 25120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:17.826924 25120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:17.826948 25120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:17.828032 25120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:17.828064 25120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:17.828090 25120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.194
I20260929 10:50:17.829526 25120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:41219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:34291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.194
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:17.829741 25120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:17.830024 25120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:17.835052 25126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:17.835052 25128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:17.835115 25125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:17.835230 25120 server_base.cc:1034] running on GCE node
I20260929 10:50:17.835538 25120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:17.835774 25120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:17.836936 25120 hybrid_clock.cc:648] HybridClock initialized: now 1790679017836930 us; error 38 us; skew 500 ppm
Sep 29 10:50:17 dist-test-slave-vqch krb5kdc[24877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679017, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:17.839341 25120 init.cc:377] Logged in from keytab as kudu/127.20.80.194@KRBTEST.COM (short username kudu)
I20260929 10:50:17.839686 25120 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:17.839753 25120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:17.839906 25120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:17.840948 25120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-1/data/instance:
uuid: "c8348d302578459787f3e1f8c4ed1c1a"
format_stamp: "Formatted at 2026-09-29 10:50:17 on dist-test-slave-vqch"
server_key: "c8a3b3f91283aca8deb722130f7e5eb5"
server_key_iv: "8b5c841ebd5d39ffa8c4b0d4fb143326"
server_key_version: "encryptionkey@0"
I20260929 10:50:17.841362 25120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance:
uuid: "c8348d302578459787f3e1f8c4ed1c1a"
format_stamp: "Formatted at 2026-09-29 10:50:17 on dist-test-slave-vqch"
server_key: "c8a3b3f91283aca8deb722130f7e5eb5"
server_key_iv: "8b5c841ebd5d39ffa8c4b0d4fb143326"
server_key_version: "encryptionkey@0"
I20260929 10:50:17.842667 25120 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:50:17.843358 25134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:17.843529 25120 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:50:17.843616 25120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-1/wal
uuid: "c8348d302578459787f3e1f8c4ed1c1a"
format_stamp: "Formatted at 2026-09-29 10:50:17 on dist-test-slave-vqch"
server_key: "c8a3b3f91283aca8deb722130f7e5eb5"
server_key_iv: "8b5c841ebd5d39ffa8c4b0d4fb143326"
server_key_version: "encryptionkey@0"
I20260929 10:50:17.843698 25120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:17.856097 25120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:17.857904 25120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:17.858057 25120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:17.858278 25120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:17.858587 25120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:17.858639 25120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:17.858672 25120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:17.858700 25120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:17.864200 25120 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.194:38465
I20260929 10:50:17.864271 25248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.194:38465 every 8 connection(s)
I20260929 10:50:17.864547 25120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
Sep 29 10:50:17 dist-test-slave-vqch krb5kdc[24877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679017, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:17.872248 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 25120
I20260929 10:50:17.872334 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance
I20260929 10:50:17.872537 20803 external_mini_cluster.cc:1428] Setting key e28999d338a98682f49d08392554749f
I20260929 10:50:17.873513 24963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:17.865328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:43945 (local address 127.20.80.254:34291)
0929 10:50:17.865407 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:17.865409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:17.865570 (+ 161us) server_negotiation.cc:407] Connection header received
0929 10:50:17.866024 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:17.866026 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:17.866035 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:17.866092 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:17.866971 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:17.867465 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:17.868412 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:17.868591 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:17.868641 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:17.870628 (+ 1987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:17.870630 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:17.870630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:17.870637 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:17.872039 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:17.872468 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:17.872469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:17.872470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:17.872534 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:17.872804 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:17.872805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:17.872805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:17.872920 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:17.873010 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:17.873302 (+ 292us) server_negotiation.cc:299] Negotiation successful
0929 10:50:17.873400 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 10:50:17.873590 25251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:17.865298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:34291 (local address 127.20.80.194:43945)
0929 10:50:17.865445 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:17.865487 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:17.865942 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:17.866143 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:17.866146 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:17.866396 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:17.866895 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:17.866900 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:17.867546 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:17.867546 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:17.868324 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:17.868326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:17.868663 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:17.868664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:17.868759 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:17.869241 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:17.869248 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:17.870543 (+ 1295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:17.872115 (+ 1572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:17.872121 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:17.872123 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:17.872376 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:17.872603 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:17.872604 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:17.872604 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:17.872731 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:17.873002 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:17.873008 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:17.873154 (+ 146us) client_negotiation.cc:769] Sending connection context
0929 10:50:17.873262 (+ 108us) client_negotiation.cc:240] Negotiation successful
0929 10:50:17.873415 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":78,"threads_started":1}
I20260929 10:50:17.874024 25249 heartbeater.cc:347] Connected to a master server at 127.20.80.254:34291
I20260929 10:50:17.874119 25249 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:17.874274 25249 heartbeater.cc:511] Master 127.20.80.254:34291 requested a full tablet report, sending...
I20260929 10:50:17.874670 24923 ts_manager.cc:194] Registered new tserver with Master: c8348d302578459787f3e1f8c4ed1c1a (127.20.80.194:38465)
I20260929 10:50:17.875166 24923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.194@KRBTEST.COM'} at 127.20.80.194:43945
WARNING: no policy specified for kudu/127.20.80.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
I20260929 10:50:17.899808 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.195:0
--local_ip_for_outbound_sockets=127.20.80.195
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:34291
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:41219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:17.983179 25254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:17.983377 25254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:17.983403 25254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:17.984509 25254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:17.984544 25254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:17.984570 25254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.195
I20260929 10:50:17.986014 25254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:41219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:34291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.195
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:17.986191 25254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:17.986364 25254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:17.991255 25260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:17.991257 25259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:17.991291 25262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:17.991314 25254 server_base.cc:1034] running on GCE node
I20260929 10:50:17.991693 25254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:17.991900 25254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:17.993038 25254 hybrid_clock.cc:648] HybridClock initialized: now 1790679017993021 us; error 26 us; skew 500 ppm
Sep 29 10:50:17 dist-test-slave-vqch krb5kdc[24877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679017, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:17.995589 25254 init.cc:377] Logged in from keytab as kudu/127.20.80.195@KRBTEST.COM (short username kudu)
I20260929 10:50:17.996140 25254 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:17.996196 25254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:17.996292 25254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:17.997413 25254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-2/data/instance:
uuid: "185ea63f11934419bef89828cf8748d1"
format_stamp: "Formatted at 2026-09-29 10:50:17 on dist-test-slave-vqch"
server_key: "f7851db3b7f3d0709d301553a3842c58"
server_key_iv: "5525d8ba30483c0e0586a1b266c83bb5"
server_key_version: "encryptionkey@0"
I20260929 10:50:17.997720 25254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance:
uuid: "185ea63f11934419bef89828cf8748d1"
format_stamp: "Formatted at 2026-09-29 10:50:17 on dist-test-slave-vqch"
server_key: "f7851db3b7f3d0709d301553a3842c58"
server_key_iv: "5525d8ba30483c0e0586a1b266c83bb5"
server_key_version: "encryptionkey@0"
I20260929 10:50:17.998955 25254 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:50:17.999751 25268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:17.999943 25254 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:18.000042 25254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-2/wal
uuid: "185ea63f11934419bef89828cf8748d1"
format_stamp: "Formatted at 2026-09-29 10:50:17 on dist-test-slave-vqch"
server_key: "f7851db3b7f3d0709d301553a3842c58"
server_key_iv: "5525d8ba30483c0e0586a1b266c83bb5"
server_key_version: "encryptionkey@0"
I20260929 10:50:18.000145 25254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:18.010268 25254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:18.012470 25254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:18.012626 25254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:18.012852 25254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:18.013239 25254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:18.013295 25254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:18.013336 25254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:18.013363 25254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:18.019157 25254 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.195:44525
I20260929 10:50:18.019209 25382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.195:44525 every 8 connection(s)
I20260929 10:50:18.019448 25254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
I20260929 10:50:18.024638 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 25254
I20260929 10:50:18.024753 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance
I20260929 10:50:18.025010 20803 external_mini_cluster.cc:1428] Setting key ddaf37999dd9fa5ab71a3f7989ae0672
Sep 29 10:50:18 dist-test-slave-vqch krb5kdc[24877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679017, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:18.028491 24963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:18.020081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:54125 (local address 127.20.80.254:34291)
0929 10:50:18.020145 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:18.020147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:18.020332 (+ 185us) server_negotiation.cc:407] Connection header received
0929 10:50:18.020763 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:18.020764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:18.020775 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:18.020827 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:18.021792 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:18.022292 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:18.023231 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:18.023415 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:18.023477 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:18.025634 (+ 2157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:18.025635 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:18.025636 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:18.025652 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:18.027102 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:18.027516 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:18.027517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:18.027518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:18.027577 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:18.027852 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:18.027853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:18.027853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:18.027970 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:18.028049 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:18.028316 (+ 267us) server_negotiation.cc:299] Negotiation successful
0929 10:50:18.028395 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 10:50:18.028692 25385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:18.020071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:34291 (local address 127.20.80.195:54125)
0929 10:50:18.020213 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:18.020246 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:18.020680 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:18.020904 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:18.020907 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:18.021222 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:18.021703 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:18.021708 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:18.022376 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:18.022376 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:18.023141 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:18.023143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:18.023508 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:18.023509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:18.023603 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:18.024019 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:18.024026 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:18.025541 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:18.027204 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:18.027211 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:18.027213 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:18.027427 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:18.027649 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:18.027650 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:18.027650 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:18.027767 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:18.028039 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:18.028046 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:18.028182 (+ 136us) client_negotiation.cc:769] Sending connection context
0929 10:50:18.028293 (+ 111us) client_negotiation.cc:240] Negotiation successful
0929 10:50:18.028458 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":86,"threads_started":1}
I20260929 10:50:18.029173 25383 heartbeater.cc:347] Connected to a master server at 127.20.80.254:34291
I20260929 10:50:18.029263 25383 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:18.029440 25383 heartbeater.cc:511] Master 127.20.80.254:34291 requested a full tablet report, sending...
I20260929 10:50:18.029789 24923 ts_manager.cc:194] Registered new tserver with Master: 185ea63f11934419bef89828cf8748d1 (127.20.80.195:44525)
I20260929 10:50:18.030215 24923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.195@KRBTEST.COM'} at 127.20.80.195:54125
I20260929 10:50:18.036756 20803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:50:18.044711 24963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:18.037984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57570 (local address 127.20.80.254:34291)
0929 10:50:18.038050 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:18.038052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:18.038132 (+ 80us) server_negotiation.cc:407] Connection header received
0929 10:50:18.038172 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:18.038173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:18.038183 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:18.038233 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:18.038741 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:18.039222 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:18.040153 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:18.040340 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:18.040400 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:18.041719 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:18.041720 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:18.041720 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:18.041726 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:18.043305 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:18.043670 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:18.043671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:18.043672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:18.043753 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:18.044006 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:18.044006 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:18.044007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:18.044223 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:18.044320 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:18.044496 (+ 176us) server_negotiation.cc:299] Negotiation successful
0929 10:50:18.044578 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 10:50:18.046772 24923 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:50:18.048297 24923 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:50:18.054625 25392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:18.050363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:35729 (local address 127.0.0.1:51740)
0929 10:50:18.050771 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:18.050781 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:18.050834 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:18.051354 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:18.051356 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:18.051358 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:18.051399 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:18.051401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:18.052632 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:18.052633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:18.053473 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:18.053476 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:18.054335 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:18.054336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:18.054399 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:18.054410 (+ 11us) client_negotiation.cc:769] Sending connection context
0929 10:50:18.054480 (+ 70us) client_negotiation.cc:240] Negotiation successful
0929 10:50:18.054516 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":63,"threads_started":1}
I20260929 10:50:18.054636 25394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:18.050455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51740 (local address 127.20.80.193:35729)
0929 10:50:18.050626 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:18.050629 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:18.051045 (+ 416us) server_negotiation.cc:407] Connection header received
0929 10:50:18.051164 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:18.051166 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:18.051185 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:18.051260 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:18.051478 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:18.052530 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:18.053568 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:18.054226 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:18.054305 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:18.054369 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:18.054508 (+ 139us) server_negotiation.cc:299] Negotiation successful
0929 10:50:18.054553 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":78,"threads_started":1}
I20260929 10:50:18.054898 25393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:18.050432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56102 (local address 127.20.80.195:44525)
0929 10:50:18.050593 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:18.050599 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:18.050765 (+ 166us) server_negotiation.cc:407] Connection header received
0929 10:50:18.050862 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:18.050864 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:18.050881 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:18.050968 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:18.051122 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:18.052229 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:18.053805 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:18.054521 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:18.054590 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:18.054643 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:18.054784 (+ 141us) server_negotiation.cc:299] Negotiation successful
0929 10:50:18.054823 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":85,"threads_started":1}
I20260929 10:50:18.054901 25395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:18.050351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:44525 (local address 127.0.0.1:56102)
0929 10:50:18.050647 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:18.050688 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:18.050768 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:18.050972 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:18.050975 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:18.050978 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:18.051041 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:18.051042 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:18.052339 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:18.052340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:18.053722 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:18.053723 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:18.054618 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:18.054619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:18.054693 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:18.054697 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:50:18.054770 (+ 73us) client_negotiation.cc:240] Negotiation successful
0929 10:50:18.054823 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":62,"thread_start_us":60,"threads_started":1}
I20260929 10:50:18.055626 25048 tablet_service.cc:1467] Processing CreateTablet for tablet cd1b443d90ed4428b8258e6c6c2deae2 (DEFAULT_TABLE table=test-table [id=1634000f1ecb41e6a38ccf8289693067]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:18.055658 25316 tablet_service.cc:1467] Processing CreateTablet for tablet cd1b443d90ed4428b8258e6c6c2deae2 (DEFAULT_TABLE table=test-table [id=1634000f1ecb41e6a38ccf8289693067]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:18.055964 25048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd1b443d90ed4428b8258e6c6c2deae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:18.055968 25316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd1b443d90ed4428b8258e6c6c2deae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:18.056377 25391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:18.050550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:38465 (local address 127.0.0.1:45562)
0929 10:50:18.050898 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:18.050907 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:18.050941 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:18.051398 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:18.051399 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:18.051401 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:18.051450 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:18.051452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:18.052323 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:18.052324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:18.053720 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:18.053723 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:18.056144 (+ 2421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:18.056145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:18.056207 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:18.056210 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:50:18.056258 (+ 48us) client_negotiation.cc:240] Negotiation successful
0929 10:50:18.056308 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":46,"thread_start_us":47,"threads_started":1}
I20260929 10:50:18.056610 25396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:18.050800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45562 (local address 127.20.80.194:38465)
0929 10:50:18.051130 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:18.051134 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:18.051175 (+ 41us) server_negotiation.cc:407] Connection header received
0929 10:50:18.051214 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:18.051216 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:18.051232 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:18.051304 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:18.051529 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:18.052214 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:18.054681 (+ 2467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:18.055984 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:18.056055 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:18.056108 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:18.056477 (+ 369us) server_negotiation.cc:299] Negotiation successful
0929 10:50:18.056521 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":84,"threads_started":1}
I20260929 10:50:18.057653 25182 tablet_service.cc:1467] Processing CreateTablet for tablet cd1b443d90ed4428b8258e6c6c2deae2 (DEFAULT_TABLE table=test-table [id=1634000f1ecb41e6a38ccf8289693067]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:18.057991 25182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd1b443d90ed4428b8258e6c6c2deae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:18.059274 25398 tablet_bootstrap.cc:492] T cd1b443d90ed4428b8258e6c6c2deae2 P c81986b3f6f54360bc43bfbf4470ac7e: Bootstrap starting.
I20260929 10:50:18.059552 25397 tablet_bootstrap.cc:492] T cd1b443d90ed4428b8258e6c6c2deae2 P 185ea63f11934419bef89828cf8748d1: Bootstrap starting.
I20260929 10:50:18.059922 25398 tablet_bootstrap.cc:654] T cd1b443d90ed4428b8258e6c6c2deae2 P c81986b3f6f54360bc43bfbf4470ac7e: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:18.060178 25397 tablet_bootstrap.cc:654] T cd1b443d90ed4428b8258e6c6c2deae2 P 185ea63f11934419bef89828cf8748d1: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:18.060469 25398 log.cc:824] T cd1b443d90ed4428b8258e6c6c2deae2 P c81986b3f6f54360bc43bfbf4470ac7e: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:18.060765 25397 log.cc:824] T cd1b443d90ed4428b8258e6c6c2deae2 P 185ea63f11934419bef89828cf8748d1: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:18.061260 25398 tablet_bootstrap.cc:492] T cd1b443d90ed4428b8258e6c6c2deae2 P c81986b3f6f54360bc43bfbf4470ac7e: No bootstrap required, opened a new log
I20260929 10:50:18.061353 25398 ts_tablet_manager.cc:1397] T cd1b443d90ed4428b8258e6c6c2deae2 P c81986b3f6f54360bc43bfbf4470ac7e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:50:18.061535 25397 tablet_bootstrap.cc:492] T cd1b443d90ed4428b8258e6c6c2deae2 P 185ea63f11934419bef89828cf8748d1: No bootstrap required, opened a new log
I20260929 10:50:18.061592 25397 ts_tablet_manager.cc:1397] T cd1b443d90ed4428b8258e6c6c2deae2 P 185ea63f11934419bef89828cf8748d1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:50:18.062973 25398 raft_consensus.cc:348] T cd1b443d90ed4428b8258e6c6c2deae2 P c81986b3f6f54360bc43bfbf4470ac7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "185ea63f11934419bef89828cf8748d1" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 44525 } } peers { permanent_uuid: "c81986b3f6f54360bc43bfbf4470ac7e" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 35729 } } peers { permanent_uuid: "c8348d302578459787f3e1f8c4ed1c1a" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 38465 } }
I20260929 10:50:18.063118 25398 raft_consensus.cc:374] T cd1b443d90ed4428b8258e6c6c2deae2 P c81986b3f6f54360bc43bfbf4470ac7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:18.063163 25398 raft_consensus.cc:729] T cd1b443d90ed4428b8258e6c6c2deae2 P c81986b3f6f54360bc43bfbf4470ac7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c81986b3f6f54360bc43bfbf4470ac7e, State: Initialized, Role: FOLLOWER
I20260929 10:50:18.063107 25397 raft_consensus.cc:348] T cd1b443d90ed4428b8258e6c6c2deae2 P 185ea63f11934419bef89828cf8748d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "185ea63f11934419bef89828cf8748d1" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 44525 } } peers { permanent_uuid: "c81986b3f6f54360bc43bfbf4470ac7e" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 35729 } } peers { permanent_uuid: "c8348d302578459787f3e1f8c4ed1c1a" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 38465 } }
I20260929 10:50:18.063289 25397 raft_consensus.cc:374] T cd1b443d90ed4428b8258e6c6c2deae2 P 185ea63f11934419bef89828cf8748d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:18.063301 25398 consensus_queue.cc:260] T cd1b443d90ed4428b8258e6c6c2deae2 P c81986b3f6f54360bc43bfbf4470ac7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "185ea63f11934419bef89828cf8748d1" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 44525 } } peers { permanent_uuid: "c81986b3f6f54360bc43bfbf4470ac7e" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 35729 } } peers { permanent_uuid: "c8348d302578459787f3e1f8c4ed1c1a" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 38465 } }
I20260929 10:50:18.063335 25397 raft_consensus.cc:729] T cd1b443d90ed4428b8258e6c6c2deae2 P 185ea63f11934419bef89828cf8748d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 185ea63f11934419bef89828cf8748d1, State: Initialized, Role: FOLLOWER
I20260929 10:50:18.063452 25397 consensus_queue.cc:260] T cd1b443d90ed4428b8258e6c6c2deae2 P 185ea63f11934419bef89828cf8748d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "185ea63f11934419bef89828cf8748d1" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 44525 } } peers { permanent_uuid: "c81986b3f6f54360bc43bfbf4470ac7e" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 35729 } } peers { permanent_uuid: "c8348d302578459787f3e1f8c4ed1c1a" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 38465 } }
I20260929 10:50:18.063585 25398 ts_tablet_manager.cc:1428] T cd1b443d90ed4428b8258e6c6c2deae2 P c81986b3f6f54360bc43bfbf4470ac7e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260929 10:50:18.063621 25115 heartbeater.cc:503] Master 127.20.80.254:34291 was elected leader, sending a full tablet report...
I20260929 10:50:18.063721 25397 ts_tablet_manager.cc:1428] T cd1b443d90ed4428b8258e6c6c2deae2 P 185ea63f11934419bef89828cf8748d1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:50:18.063727 25383 heartbeater.cc:503] Master 127.20.80.254:34291 was elected leader, sending a full tablet report...
I20260929 10:50:18.063911 25401 tablet_bootstrap.cc:492] T cd1b443d90ed4428b8258e6c6c2deae2 P c8348d302578459787f3e1f8c4ed1c1a: Bootstrap starting.
I20260929 10:50:18.064561 25401 tablet_bootstrap.cc:654] T cd1b443d90ed4428b8258e6c6c2deae2 P c8348d302578459787f3e1f8c4ed1c1a: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:18.065150 25401 log.cc:824] T cd1b443d90ed4428b8258e6c6c2deae2 P c8348d302578459787f3e1f8c4ed1c1a: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:18.065912 25401 tablet_bootstrap.cc:492] T cd1b443d90ed4428b8258e6c6c2deae2 P c8348d302578459787f3e1f8c4ed1c1a: No bootstrap required, opened a new log
I20260929 10:50:18.065964 25401 ts_tablet_manager.cc:1397] T cd1b443d90ed4428b8258e6c6c2deae2 P c8348d302578459787f3e1f8c4ed1c1a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:50:18.067266 25401 raft_consensus.cc:348] T cd1b443d90ed4428b8258e6c6c2deae2 P c8348d302578459787f3e1f8c4ed1c1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "185ea63f11934419bef89828cf8748d1" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 44525 } } peers { permanent_uuid: "c81986b3f6f54360bc43bfbf4470ac7e" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 35729 } } peers { permanent_uuid: "c8348d302578459787f3e1f8c4ed1c1a" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 38465 } }
I20260929 10:50:18.067399 25401 raft_consensus.cc:374] T cd1b443d90ed4428b8258e6c6c2deae2 P c8348d302578459787f3e1f8c4ed1c1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:18.067420 25401 raft_consensus.cc:729] T cd1b443d90ed4428b8258e6c6c2deae2 P c8348d302578459787f3e1f8c4ed1c1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8348d302578459787f3e1f8c4ed1c1a, State: Initialized, Role: FOLLOWER
I20260929 10:50:18.067534 25401 consensus_queue.cc:260] T cd1b443d90ed4428b8258e6c6c2deae2 P c8348d302578459787f3e1f8c4ed1c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "185ea63f11934419bef89828cf8748d1" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 44525 } } peers { permanent_uuid: "c81986b3f6f54360bc43bfbf4470ac7e" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 35729 } } peers { permanent_uuid: "c8348d302578459787f3e1f8c4ed1c1a" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 38465 } }
I20260929 10:50:18.067785 25401 ts_tablet_manager.cc:1428] T cd1b443d90ed4428b8258e6c6c2deae2 P c8348d302578459787f3e1f8c4ed1c1a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:50:18.067852 25249 heartbeater.cc:503] Master 127.20.80.254:34291 was elected leader, sending a full tablet report...
W20260929 10:50:18.115070 25250 tablet.cc:2369] T cd1b443d90ed4428b8258e6c6c2deae2 P c8348d302578459787f3e1f8c4ed1c1a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:50:18.216228 25405 raft_consensus.cc:482] T cd1b443d90ed4428b8258e6c6c2deae2 P c8348d302578459787f3e1f8c4ed1c1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:50:18.216360 25405 raft_consensus.cc:504] T cd1b443d90ed4428b8258e6c6c2deae2 P c8348d302578459787f3e1f8c4ed1c1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "185ea63f11934419bef89828cf8748d1" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 44525 } } peers { permanent_uuid: "c81986b3f6f54360bc43bfbf4470ac7e" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 35729 } } peers { permanent_uuid: "c8348d302578459787f3e1f8c4ed1c1a" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 38465 } }
I20260929 10:50:18.216848 25405 leader_election.cc:288] T cd1b443d90ed4428b8258e6c6c2deae2 P c8348d302578459787f3e1f8c4ed1c1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 185ea63f11934419bef89828cf8748d1 (127.20.80.195:44525), c81986b3f6f54360bc43bfbf4470ac7e (127.20.80.193:35729)
I20260929 10:50:18.219648 25393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:18.217111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:38357 (local address 127.20.80.195:44525)
0929 10:50:18.217152 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:18.217154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:18.217172 (+ 18us) server_negotiation.cc:407] Connection header received
0929 10:50:18.217255 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:18.217256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:18.217264 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:18.217330 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:18.217508 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:18.217979 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:18.218902 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:18.219266 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:18.219389 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:18.219399 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:18.219557 (+ 158us) server_negotiation.cc:299] Negotiation successful
0929 10:50:18.219592 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260929 10:50:18.219647 25251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:18.217065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:44525 (local address 127.20.80.194:38357)
0929 10:50:18.217116 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:18.217130 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:18.217181 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:18.217353 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:18.217354 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:18.217356 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:18.217434 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:18.217436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:18.218061 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:18.218062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:18.218803 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:18.218805 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:18.219389 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:18.219389 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:18.219474 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:18.219477 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:50:18.219539 (+ 62us) client_negotiation.cc:240] Negotiation successful
0929 10:50:18.219577 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260929 10:50:18.220036 25336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1b443d90ed4428b8258e6c6c2deae2" candidate_uuid: "c8348d302578459787f3e1f8c4ed1c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "185ea63f11934419bef89828cf8748d1" is_pre_election: true
I20260929 10:50:18.220196 25394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:18.217126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:49077 (local address 127.20.80.193:35729)
0929 10:50:18.217174 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:18.217175 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:18.217522 (+ 347us) server_negotiation.cc:407] Connection header received
0929 10:50:18.217596 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:18.217597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:18.217607 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:18.217683 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:18.217854 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:18.218360 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:18.219341 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:18.219824 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:18.219902 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:18.219909 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:18.220086 (+ 177us) server_negotiation.cc:299] Negotiation successful
0929 10:50:18.220124 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 10:50:18.220201 25336 raft_consensus.cc:2454] T cd1b443d90ed4428b8258e6c6c2deae2 P 185ea63f11934419bef89828cf8748d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8348d302578459787f3e1f8c4ed1c1a in term 0.
I20260929 10:50:18.220206 25406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:18.217069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:35729 (local address 127.20.80.194:49077)
0929 10:50:18.217441 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:18.217450 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:18.217527 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:18.217686 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:18.217686 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:18.217688 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:18.217775 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:18.217777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:18.218447 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:18.218448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:18.219242 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:18.219244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:18.219926 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:18.219928 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:18.220000 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:18.220004 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:50:18.220079 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 10:50:18.220121 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"mutex_wait_us":186,"thread_start_us":116,"threads_started":1}
I20260929 10:50:18.220439 25135 leader_election.cc:302] T cd1b443d90ed4428b8258e6c6c2deae2 P c8348d302578459787f3e1f8c4ed1c1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 185ea63f11934419bef89828cf8748d1, c8348d302578459787f3e1f8c4ed1c1a; no voters:
I20260929 10:50:18.220582 25405 raft_consensus.cc:2781] T cd1b443d90ed4428b8258e6c6c2deae2 P c8348d302578459787f3e1f8c4ed1c1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:50:18.220559 25069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1b443d90ed4428b8258e6c6c2deae2" candidate_uuid: "c8348d302578459787f3e1f8c4ed1c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c81986b3f6f54360bc43bfbf4470ac7e" is_pre_election: true
I20260929 10:50:18.220651 25405 raft_consensus.cc:482] T cd1b443d90ed4428b8258e6c6c2deae2 P c8348d302578459787f3e1f8c4ed1c1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:50:18.220676 25405 raft_consensus.cc:3037] T cd1b443d90ed4428b8258e6c6c2deae2 P c8348d302578459787f3e1f8c4ed1c1a [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:18.220715 25069 raft_consensus.cc:2454] T cd1b443d90ed4428b8258e6c6c2deae2 P c81986b3f6f54360bc43bfbf4470ac7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8348d302578459787f3e1f8c4ed1c1a in term 0.
W20260929 10:50:18.221108 25116 tablet.cc:2369] T cd1b443d90ed4428b8258e6c6c2deae2 P c81986b3f6f54360bc43bfbf4470ac7e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:50:18.221421 25405 raft_consensus.cc:504] T cd1b443d90ed4428b8258e6c6c2deae2 P c8348d302578459787f3e1f8c4ed1c1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "185ea63f11934419bef89828cf8748d1" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 44525 } } peers { permanent_uuid: "c81986b3f6f54360bc43bfbf4470ac7e" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 35729 } } peers { permanent_uuid: "c8348d302578459787f3e1f8c4ed1c1a" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 38465 } }
I20260929 10:50:18.221549 25405 leader_election.cc:288] T cd1b443d90ed4428b8258e6c6c2deae2 P c8348d302578459787f3e1f8c4ed1c1a [CANDIDATE]: Term 1 election: Requested vote from peers 185ea63f11934419bef89828cf8748d1 (127.20.80.195:44525), c81986b3f6f54360bc43bfbf4470ac7e (127.20.80.193:35729)
I20260929 10:50:18.221704 25069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1b443d90ed4428b8258e6c6c2deae2" candidate_uuid: "c8348d302578459787f3e1f8c4ed1c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c81986b3f6f54360bc43bfbf4470ac7e"
I20260929 10:50:18.221691 25336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1b443d90ed4428b8258e6c6c2deae2" candidate_uuid: "c8348d302578459787f3e1f8c4ed1c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "185ea63f11934419bef89828cf8748d1"
I20260929 10:50:18.221778 25069 raft_consensus.cc:3037] T cd1b443d90ed4428b8258e6c6c2deae2 P c81986b3f6f54360bc43bfbf4470ac7e [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:18.221801 25336 raft_consensus.cc:3037] T cd1b443d90ed4428b8258e6c6c2deae2 P 185ea63f11934419bef89828cf8748d1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:18.222489 25069 raft_consensus.cc:2454] T cd1b443d90ed4428b8258e6c6c2deae2 P c81986b3f6f54360bc43bfbf4470ac7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8348d302578459787f3e1f8c4ed1c1a in term 1.
I20260929 10:50:18.222601 25336 raft_consensus.cc:2454] T cd1b443d90ed4428b8258e6c6c2deae2 P 185ea63f11934419bef89828cf8748d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8348d302578459787f3e1f8c4ed1c1a in term 1.
I20260929 10:50:18.222661 25137 leader_election.cc:302] T cd1b443d90ed4428b8258e6c6c2deae2 P c8348d302578459787f3e1f8c4ed1c1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c81986b3f6f54360bc43bfbf4470ac7e, c8348d302578459787f3e1f8c4ed1c1a; no voters:
I20260929 10:50:18.222772 25405 raft_consensus.cc:2781] T cd1b443d90ed4428b8258e6c6c2deae2 P c8348d302578459787f3e1f8c4ed1c1a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:18.222931 25405 raft_consensus.cc:686] T cd1b443d90ed4428b8258e6c6c2deae2 P c8348d302578459787f3e1f8c4ed1c1a [term 1 LEADER]: Becoming Leader. State: Replica: c8348d302578459787f3e1f8c4ed1c1a, State: Running, Role: LEADER
I20260929 10:50:18.223022 25405 consensus_queue.cc:237] T cd1b443d90ed4428b8258e6c6c2deae2 P c8348d302578459787f3e1f8c4ed1c1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "185ea63f11934419bef89828cf8748d1" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 44525 } } peers { permanent_uuid: "c81986b3f6f54360bc43bfbf4470ac7e" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 35729 } } peers { permanent_uuid: "c8348d302578459787f3e1f8c4ed1c1a" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 38465 } }
I20260929 10:50:18.223910 24923 catalog_manager.cc:5495] T cd1b443d90ed4428b8258e6c6c2deae2 P c8348d302578459787f3e1f8c4ed1c1a reported cstate change: term changed from 0 to 1, leader changed from <none> to c8348d302578459787f3e1f8c4ed1c1a (127.20.80.194). New cstate: current_term: 1 leader_uuid: "c8348d302578459787f3e1f8c4ed1c1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "185ea63f11934419bef89828cf8748d1" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 44525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c81986b3f6f54360bc43bfbf4470ac7e" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 35729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8348d302578459787f3e1f8c4ed1c1a" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 38465 } health_report { overall_health: HEALTHY } } }
I20260929 10:50:18.239301 25396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:18.236754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45576 (local address 127.20.80.194:38465)
0929 10:50:18.236805 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:18.236807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:18.236835 (+ 28us) server_negotiation.cc:407] Connection header received
0929 10:50:18.236885 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:18.236886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:18.236900 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:18.236952 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:50:18.237237 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:18.237754 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:18.238534 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:18.238729 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:18.238804 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:18.238959 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:50:18.239057 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:50:18.239151 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:18.239198 (+ 47us) server_negotiation.cc:299] Negotiation successful
0929 10:50:18.239218 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260929 10:50:18.241086 25069 raft_consensus.cc:1261] T cd1b443d90ed4428b8258e6c6c2deae2 P c81986b3f6f54360bc43bfbf4470ac7e [term 1 FOLLOWER]: Refusing update from remote peer c8348d302578459787f3e1f8c4ed1c1a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:18.241241 25336 raft_consensus.cc:1261] T cd1b443d90ed4428b8258e6c6c2deae2 P 185ea63f11934419bef89828cf8748d1 [term 1 FOLLOWER]: Refusing update from remote peer c8348d302578459787f3e1f8c4ed1c1a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:18.241457 25407 consensus_queue.cc:1035] T cd1b443d90ed4428b8258e6c6c2deae2 P c8348d302578459787f3e1f8c4ed1c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c81986b3f6f54360bc43bfbf4470ac7e" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 35729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:50:18.241586 25405 consensus_queue.cc:1035] T cd1b443d90ed4428b8258e6c6c2deae2 P c8348d302578459787f3e1f8c4ed1c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "185ea63f11934419bef89828cf8748d1" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 44525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:50:18.243999 25410 mvcc.cc:205] Tried to move back new op lower bound from 7334621258712502272 to 7334621258642292736. Current Snapshot: MvccSnapshot[applied={T|T < 7334621258712502272}]
I20260929 10:50:18.246404 24923 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:50:18.246500 24923 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:50:18.247242 24923 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f473f2db0e09439d8c96972c5357949a: Sending DeleteTablet for 3 replicas of tablet cd1b443d90ed4428b8258e6c6c2deae2
I20260929 10:50:18.247617 25316 tablet_service.cc:1514] Processing DeleteTablet for tablet cd1b443d90ed4428b8258e6c6c2deae2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:18 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:56102
I20260929 10:50:18.247658 25048 tablet_service.cc:1514] Processing DeleteTablet for tablet cd1b443d90ed4428b8258e6c6c2deae2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:18 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:51740
I20260929 10:50:18.247634 25182 tablet_service.cc:1514] Processing DeleteTablet for tablet cd1b443d90ed4428b8258e6c6c2deae2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:18 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:45562
I20260929 10:50:18.247882 25419 tablet_replica.cc:330] T cd1b443d90ed4428b8258e6c6c2deae2 P 185ea63f11934419bef89828cf8748d1: stopping tablet replica
I20260929 10:50:18.248039 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 24985
I20260929 10:50:18.248164 25420 tablet_replica.cc:330] T cd1b443d90ed4428b8258e6c6c2deae2 P c8348d302578459787f3e1f8c4ed1c1a: stopping tablet replica
I20260929 10:50:18.249699 25420 raft_consensus.cc:2229] T cd1b443d90ed4428b8258e6c6c2deae2 P c8348d302578459787f3e1f8c4ed1c1a [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:50:18.249806 25420 raft_consensus.cc:2258] T cd1b443d90ed4428b8258e6c6c2deae2 P c8348d302578459787f3e1f8c4ed1c1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:18.249958 25419 raft_consensus.cc:2229] T cd1b443d90ed4428b8258e6c6c2deae2 P 185ea63f11934419bef89828cf8748d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:50:18.250072 25419 raft_consensus.cc:2258] T cd1b443d90ed4428b8258e6c6c2deae2 P 185ea63f11934419bef89828cf8748d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:18.250348 25420 ts_tablet_manager.cc:1905] T cd1b443d90ed4428b8258e6c6c2deae2 P c8348d302578459787f3e1f8c4ed1c1a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:18.250504 25419 ts_tablet_manager.cc:1905] T cd1b443d90ed4428b8258e6c6c2deae2 P 185ea63f11934419bef89828cf8748d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:18.252066 25420 ts_tablet_manager.cc:1918] T cd1b443d90ed4428b8258e6c6c2deae2 P c8348d302578459787f3e1f8c4ed1c1a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:50:18.252198 25420 log.cc:1196] T cd1b443d90ed4428b8258e6c6c2deae2 P c8348d302578459787f3e1f8c4ed1c1a: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-1/wal/wals/cd1b443d90ed4428b8258e6c6c2deae2
I20260929 10:50:18.252562 25420 ts_tablet_manager.cc:1939] T cd1b443d90ed4428b8258e6c6c2deae2 P c8348d302578459787f3e1f8c4ed1c1a: Deleting consensus metadata
I20260929 10:50:18.252559 25419 ts_tablet_manager.cc:1918] T cd1b443d90ed4428b8258e6c6c2deae2 P 185ea63f11934419bef89828cf8748d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:50:18.252626 25419 log.cc:1196] T cd1b443d90ed4428b8258e6c6c2deae2 P 185ea63f11934419bef89828cf8748d1: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790678996126930-20803-0/minicluster-data/ts-2/wal/wals/cd1b443d90ed4428b8258e6c6c2deae2
I20260929 10:50:18.252921 25419 ts_tablet_manager.cc:1939] T cd1b443d90ed4428b8258e6c6c2deae2 P 185ea63f11934419bef89828cf8748d1: Deleting consensus metadata
W20260929 10:50:18.255309 24908 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.80.193:35729 (error 108)
I20260929 10:50:18.255620 24906 catalog_manager.cc:4841] TS 185ea63f11934419bef89828cf8748d1 (127.20.80.195:44525): tablet cd1b443d90ed4428b8258e6c6c2deae2 (table test-table [id=1634000f1ecb41e6a38ccf8289693067]) successfully deleted
I20260929 10:50:18.255723 24906 catalog_manager.cc:4841] TS c8348d302578459787f3e1f8c4ed1c1a (127.20.80.194:38465): tablet cd1b443d90ed4428b8258e6c6c2deae2 (table test-table [id=1634000f1ecb41e6a38ccf8289693067]) successfully deleted
W20260929 10:50:18.256119 25391 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:18.255739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:35729 (local address 127.0.0.1:51750)
0929 10:50:18.255797 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:18.255810 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:18.255850 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:18.256054 (+ 204us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.193:35729: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":36}
W20260929 10:50:18.256242 24908 catalog_manager.cc:4584] TS c81986b3f6f54360bc43bfbf4470ac7e (127.20.80.193:35729): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cd1b443d90ed4428b8258e6c6c2deae2: Network error: Client connection negotiation failed: client connection to 127.20.80.193:35729: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 10:50:18.256351 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 25120
I20260929 10:50:18.262236 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 25254
I20260929 10:50:18.267133 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 24891
2026-09-29T10:50:18Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2024 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-29T10:50:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:50:18Z Disabled control of system clock
I20260929 10:50:18.295403 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:38235
--webserver_interface=127.20.80.254
--webserver_port=0
--builtin_ntp_servers=127.20.80.212:43009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:38235
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 10:50:18.374941 25426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:18.375136 25426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:18.375156 25426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:18.376338 25426 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:50:18.376376 25426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:18.376389 25426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:18.376401 25426 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:50:18.376413 25426 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:50:18.377758 25426 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:43009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:38235
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:38235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:18.377955 25426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:18.378185 25426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:18.383031 25434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:18.383033 25432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:18.383131 25431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:18.383297 25426 server_base.cc:1034] running on GCE node
I20260929 10:50:18.383553 25426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:18.383827 25426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:18.384999 25426 hybrid_clock.cc:648] HybridClock initialized: now 1790679018384954 us; error 33 us; skew 500 ppm
I20260929 10:50:18.386206 25426 webserver.cc:460] Webserver started at http://127.20.80.254:43561/ using document root <none> and password file <none>
I20260929 10:50:18.386458 25426 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:18.386507 25426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:18.386654 25426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:18.387745 25426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "261c150f414d4f2c9d00d8615d75f02e"
format_stamp: "Formatted at 2026-09-29 10:50:18 on dist-test-slave-vqch"
server_key: "6484f69d8c49c45ae04bdd3ccb05b45f"
server_key_iv: "bdeef7cb63a904c20f8055eee187f88e"
server_key_version: "encryptionkey@0"
I20260929 10:50:18.388099 25426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "261c150f414d4f2c9d00d8615d75f02e"
format_stamp: "Formatted at 2026-09-29 10:50:18 on dist-test-slave-vqch"
server_key: "6484f69d8c49c45ae04bdd3ccb05b45f"
server_key_iv: "bdeef7cb63a904c20f8055eee187f88e"
server_key_version: "encryptionkey@0"
I20260929 10:50:18.389433 25426 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:50:18.390065 25440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:18.390249 25426 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:18.390336 25426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "261c150f414d4f2c9d00d8615d75f02e"
format_stamp: "Formatted at 2026-09-29 10:50:18 on dist-test-slave-vqch"
server_key: "6484f69d8c49c45ae04bdd3ccb05b45f"
server_key_iv: "bdeef7cb63a904c20f8055eee187f88e"
server_key_version: "encryptionkey@0"
I20260929 10:50:18.390414 25426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:18.393669 25426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:18.393922 25426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:18.394061 25426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:18.397897 25426 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:38235
I20260929 10:50:18.397966 25493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:38235 every 8 connection(s)
I20260929 10:50:18.398277 25426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:50:18.398932 25494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:18.399335 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 25426
I20260929 10:50:18.399435 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:50:18.399642 20803 external_mini_cluster.cc:1428] Setting key 4eaedcb7a663ee70ca61f716e12f9e75
I20260929 10:50:18.402813 25497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:18.400431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57362 (local address 127.20.80.254:38235)
0929 10:50:18.400642 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:18.400651 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:18.400684 (+ 33us) server_negotiation.cc:407] Connection header received
0929 10:50:18.401443 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:18.401452 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:18.401507 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:18.401650 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:18.402339 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:18.402342 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:18.402342 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:50:18.402352 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:18.402396 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:50:18.402406 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:18.402461 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:18.402586 (+ 125us) server_negotiation.cc:299] Negotiation successful
0929 10:50:18.402649 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":83,"threads_started":1}
I20260929 10:50:18.403280 25494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 261c150f414d4f2c9d00d8615d75f02e: Bootstrap starting.
I20260929 10:50:18.403856 25494 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 261c150f414d4f2c9d00d8615d75f02e: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:18.404124 25494 log.cc:824] T 00000000000000000000000000000000 P 261c150f414d4f2c9d00d8615d75f02e: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:18.404752 25494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 261c150f414d4f2c9d00d8615d75f02e: No bootstrap required, opened a new log
I20260929 10:50:18.406106 25494 raft_consensus.cc:348] T 00000000000000000000000000000000 P 261c150f414d4f2c9d00d8615d75f02e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "261c150f414d4f2c9d00d8615d75f02e" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 38235 } }
I20260929 10:50:18.406235 25494 raft_consensus.cc:374] T 00000000000000000000000000000000 P 261c150f414d4f2c9d00d8615d75f02e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:18.406302 25494 raft_consensus.cc:729] T 00000000000000000000000000000000 P 261c150f414d4f2c9d00d8615d75f02e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 261c150f414d4f2c9d00d8615d75f02e, State: Initialized, Role: FOLLOWER
I20260929 10:50:18.406431 25494 consensus_queue.cc:260] T 00000000000000000000000000000000 P 261c150f414d4f2c9d00d8615d75f02e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "261c150f414d4f2c9d00d8615d75f02e" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 38235 } }
I20260929 10:50:18.406503 25494 raft_consensus.cc:388] T 00000000000000000000000000000000 P 261c150f414d4f2c9d00d8615d75f02e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:50:18.406543 25494 raft_consensus.cc:482] T 00000000000000000000000000000000 P 261c150f414d4f2c9d00d8615d75f02e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:50:18.406592 25494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 261c150f414d4f2c9d00d8615d75f02e [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:18.438700 25494 raft_consensus.cc:504] T 00000000000000000000000000000000 P 261c150f414d4f2c9d00d8615d75f02e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "261c150f414d4f2c9d00d8615d75f02e" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 38235 } }
I20260929 10:50:18.438925 25494 leader_election.cc:302] T 00000000000000000000000000000000 P 261c150f414d4f2c9d00d8615d75f02e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 261c150f414d4f2c9d00d8615d75f02e; no voters:
I20260929 10:50:18.439186 25494 leader_election.cc:288] T 00000000000000000000000000000000 P 261c150f414d4f2c9d00d8615d75f02e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:50:18.439360 25499 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 261c150f414d4f2c9d00d8615d75f02e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:18.439524 25494 sys_catalog.cc:564] T 00000000000000000000000000000000 P 261c150f414d4f2c9d00d8615d75f02e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:50:18.439560 25499 raft_consensus.cc:686] T 00000000000000000000000000000000 P 261c150f414d4f2c9d00d8615d75f02e [term 1 LEADER]: Becoming Leader. State: Replica: 261c150f414d4f2c9d00d8615d75f02e, State: Running, Role: LEADER
I20260929 10:50:18.439728 25499 consensus_queue.cc:237] T 00000000000000000000000000000000 P 261c150f414d4f2c9d00d8615d75f02e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "261c150f414d4f2c9d00d8615d75f02e" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 38235 } }
I20260929 10:50:18.440173 25500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 261c150f414d4f2c9d00d8615d75f02e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "261c150f414d4f2c9d00d8615d75f02e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "261c150f414d4f2c9d00d8615d75f02e" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 38235 } } }
I20260929 10:50:18.440330 25500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 261c150f414d4f2c9d00d8615d75f02e [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:18.440210 25501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 261c150f414d4f2c9d00d8615d75f02e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 261c150f414d4f2c9d00d8615d75f02e. Latest consensus state: current_term: 1 leader_uuid: "261c150f414d4f2c9d00d8615d75f02e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "261c150f414d4f2c9d00d8615d75f02e" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 38235 } } }
I20260929 10:50:18.440532 25501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 261c150f414d4f2c9d00d8615d75f02e [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:18.440687 25508 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:50:18.441316 25508 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:50:18.443320 25508 catalog_manager.cc:1347] Generated new cluster ID: 4a3fdc492f32488b90475866b9f03587
I20260929 10:50:18.443368 25508 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:50:18.467886 25508 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:50:18.468096 25508 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:50:18.476830 25508 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 261c150f414d4f2c9d00d8615d75f02e: Generated new TSK 0
I20260929 10:50:18.477061 25508 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:50:18.481722 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.193:0
--local_ip_for_outbound_sockets=127.20.80.193
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:38235
--builtin_ntp_servers=127.20.80.212:43009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 10:50:18.566315 25518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:18.566517 25518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:18.566538 25518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:18.567741 25518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:18.567780 25518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:18.567806 25518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.193
I20260929 10:50:18.569547 25518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:43009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:38235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.193
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:18.569816 25518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:18.570116 25518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:18.575382 25524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:18.575366 25523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:18.576038 25526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:18.576038 25518 server_base.cc:1034] running on GCE node
I20260929 10:50:18.576337 25518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:18.576606 25518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:18.577769 25518 hybrid_clock.cc:648] HybridClock initialized: now 1790679018577759 us; error 35 us; skew 500 ppm
I20260929 10:50:18.578994 25518 webserver.cc:460] Webserver started at http://127.20.80.193:44837/ using document root <none> and password file <none>
I20260929 10:50:18.579237 25518 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:18.579305 25518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:18.579452 25518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:18.580514 25518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-0/data/instance:
uuid: "f9a98ddaa95142199c072d4b5b287bd6"
format_stamp: "Formatted at 2026-09-29 10:50:18 on dist-test-slave-vqch"
server_key: "cbb94f134db4cc5b86259c9c220efc4f"
server_key_iv: "d6f0cafed68cd737f45a6ceff005ece4"
server_key_version: "encryptionkey@0"
I20260929 10:50:18.580892 25518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance:
uuid: "f9a98ddaa95142199c072d4b5b287bd6"
format_stamp: "Formatted at 2026-09-29 10:50:18 on dist-test-slave-vqch"
server_key: "cbb94f134db4cc5b86259c9c220efc4f"
server_key_iv: "d6f0cafed68cd737f45a6ceff005ece4"
server_key_version: "encryptionkey@0"
I20260929 10:50:18.582262 25518 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:50:18.583005 25532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:18.583181 25518 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:50:18.583278 25518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-0/wal
uuid: "f9a98ddaa95142199c072d4b5b287bd6"
format_stamp: "Formatted at 2026-09-29 10:50:18 on dist-test-slave-vqch"
server_key: "cbb94f134db4cc5b86259c9c220efc4f"
server_key_iv: "d6f0cafed68cd737f45a6ceff005ece4"
server_key_version: "encryptionkey@0"
I20260929 10:50:18.583364 25518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:18.586787 25518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:18.587049 25518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:18.587189 25518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:18.587393 25518 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:18.587692 25518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:18.587750 25518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:18.587780 25518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:18.587800 25518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:18.593240 25518 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.193:39881
I20260929 10:50:18.593309 25645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.193:39881 every 8 connection(s)
I20260929 10:50:18.593609 25518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
I20260929 10:50:18.596431 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 25518
I20260929 10:50:18.596513 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance
I20260929 10:50:18.596721 20803 external_mini_cluster.cc:1428] Setting key e1936539679ee671ac0fb6b60824d665
I20260929 10:50:18.596894 25497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:18.594363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:47281 (local address 127.20.80.254:38235)
0929 10:50:18.594420 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:18.594422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:18.594631 (+ 209us) server_negotiation.cc:407] Connection header received
0929 10:50:18.595128 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:18.595129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:18.595136 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:18.595189 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:18.596583 (+ 1394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:18.596585 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:18.596585 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:50:18.596590 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:18.596609 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:50:18.596614 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:18.596675 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:18.596802 (+ 127us) server_negotiation.cc:299] Negotiation successful
0929 10:50:18.596817 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260929 10:50:18.597105 25648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:18.594363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:38235 (local address 127.20.80.193:47281)
0929 10:50:18.594524 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:18.594553 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:18.595024 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:18.595273 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:18.595277 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:18.595289 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:18.596158 (+ 869us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:50:18.596159 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:18.596194 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:50:18.596221 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:18.596682 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:18.596688 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:18.596692 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:50:18.596781 (+ 89us) client_negotiation.cc:240] Negotiation successful
0929 10:50:18.596848 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":75,"threads_started":1}
I20260929 10:50:18.597666 25646 heartbeater.cc:347] Connected to a master server at 127.20.80.254:38235
I20260929 10:50:18.597743 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.194:0
--local_ip_for_outbound_sockets=127.20.80.194
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:38235
--builtin_ntp_servers=127.20.80.212:43009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260929 10:50:18.597761 25646 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:18.597918 25646 heartbeater.cc:511] Master 127.20.80.254:38235 requested a full tablet report, sending...
I20260929 10:50:18.598428 25457 ts_manager.cc:194] Registered new tserver with Master: f9a98ddaa95142199c072d4b5b287bd6 (127.20.80.193:39881)
W20260929 10:50:18.679152 25649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:18.679345 25649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:18.679374 25649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:18.680557 25649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:18.680595 25649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:18.680621 25649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.194
I20260929 10:50:18.682077 25649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:43009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:38235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.194
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:18.682317 25649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:18.682657 25649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:18.687714 25657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:18.687763 25655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:18.687680 25654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:18.687858 25649 server_base.cc:1034] running on GCE node
I20260929 10:50:18.688165 25649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:18.688431 25649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:18.689594 25649 hybrid_clock.cc:648] HybridClock initialized: now 1790679018689568 us; error 38 us; skew 500 ppm
I20260929 10:50:18.690963 25649 webserver.cc:460] Webserver started at http://127.20.80.194:46387/ using document root <none> and password file <none>
I20260929 10:50:18.691239 25649 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:18.691318 25649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:18.691466 25649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:18.692646 25649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-1/data/instance:
uuid: "015069fa88dd42f79301cce68f68b787"
format_stamp: "Formatted at 2026-09-29 10:50:18 on dist-test-slave-vqch"
server_key: "c54bea6c41a3260fd52628d4f9442ecb"
server_key_iv: "32e9d7309298534dfff95e74ad75454f"
server_key_version: "encryptionkey@0"
I20260929 10:50:18.693048 25649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance:
uuid: "015069fa88dd42f79301cce68f68b787"
format_stamp: "Formatted at 2026-09-29 10:50:18 on dist-test-slave-vqch"
server_key: "c54bea6c41a3260fd52628d4f9442ecb"
server_key_iv: "32e9d7309298534dfff95e74ad75454f"
server_key_version: "encryptionkey@0"
I20260929 10:50:18.694427 25649 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:50:18.695122 25663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:18.695271 25649 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:50:18.695367 25649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-1/wal
uuid: "015069fa88dd42f79301cce68f68b787"
format_stamp: "Formatted at 2026-09-29 10:50:18 on dist-test-slave-vqch"
server_key: "c54bea6c41a3260fd52628d4f9442ecb"
server_key_iv: "32e9d7309298534dfff95e74ad75454f"
server_key_version: "encryptionkey@0"
I20260929 10:50:18.695443 25649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:18.698844 25649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:18.699091 25649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:18.699232 25649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:18.699433 25649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:18.699733 25649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:18.699792 25649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:18.699831 25649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:18.699857 25649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:18.705574 25649 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.194:36005
I20260929 10:50:18.705662 25778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.194:36005 every 8 connection(s)
I20260929 10:50:18.705943 25649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
I20260929 10:50:18.709206 25497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:18.706609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:60935 (local address 127.20.80.254:38235)
0929 10:50:18.706654 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:18.706657 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:18.706926 (+ 269us) server_negotiation.cc:407] Connection header received
0929 10:50:18.707351 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:18.707352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:18.707358 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:18.707416 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:18.708947 (+ 1531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:18.708948 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:18.708948 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:50:18.708952 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:18.708967 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:50:18.708971 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:18.709072 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:18.709139 (+ 67us) server_negotiation.cc:299] Negotiation successful
0929 10:50:18.709152 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 10:50:18.709393 25781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:18.706619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:38235 (local address 127.20.80.194:60935)
0929 10:50:18.706816 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:18.706846 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:18.707261 (+ 415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:18.707969 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:18.707972 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:18.707980 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:18.708816 (+ 836us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:50:18.708816 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:18.708842 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:50:18.708866 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:18.709071 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:18.709075 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:18.709077 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 10:50:18.709134 (+ 57us) client_negotiation.cc:240] Negotiation successful
0929 10:50:18.709180 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":84,"threads_started":1}
I20260929 10:50:18.709899 25779 heartbeater.cc:347] Connected to a master server at 127.20.80.254:38235
I20260929 10:50:18.709971 25779 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:18.710131 25779 heartbeater.cc:511] Master 127.20.80.254:38235 requested a full tablet report, sending...
I20260929 10:50:18.710520 25457 ts_manager.cc:194] Registered new tserver with Master: 015069fa88dd42f79301cce68f68b787 (127.20.80.194:36005)
I20260929 10:50:18.712404 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 25649
I20260929 10:50:18.712517 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance
I20260929 10:50:18.712697 20803 external_mini_cluster.cc:1428] Setting key ef61c0466b890c25ff0c02fed36e04e1
I20260929 10:50:18.713649 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.195:0
--local_ip_for_outbound_sockets=127.20.80.195
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:38235
--builtin_ntp_servers=127.20.80.212:43009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 10:50:18.798153 25782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:18.798348 25782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:18.798370 25782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:18.799634 25782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:18.799676 25782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:18.799736 25782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.195
I20260929 10:50:18.801119 25782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:43009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:38235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.195
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:18.801362 25782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:18.801663 25782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:18.806941 25788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:18.806980 25787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:18.806941 25790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:18.807298 25782 server_base.cc:1034] running on GCE node
I20260929 10:50:18.808139 25782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:18.808399 25782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:18.809538 25782 hybrid_clock.cc:648] HybridClock initialized: now 1790679018809512 us; error 38 us; skew 500 ppm
I20260929 10:50:18.815148 25782 webserver.cc:460] Webserver started at http://127.20.80.195:45003/ using document root <none> and password file <none>
I20260929 10:50:18.815423 25782 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:18.815472 25782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:18.815645 25782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:18.816754 25782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-2/data/instance:
uuid: "3881f10a1d574f4ba23c983a4f686118"
format_stamp: "Formatted at 2026-09-29 10:50:18 on dist-test-slave-vqch"
server_key: "f7526172778f2207a74b81623905d408"
server_key_iv: "ec96f5b43ce605c08af1fa6361d37bf0"
server_key_version: "encryptionkey@0"
I20260929 10:50:18.817142 25782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance:
uuid: "3881f10a1d574f4ba23c983a4f686118"
format_stamp: "Formatted at 2026-09-29 10:50:18 on dist-test-slave-vqch"
server_key: "f7526172778f2207a74b81623905d408"
server_key_iv: "ec96f5b43ce605c08af1fa6361d37bf0"
server_key_version: "encryptionkey@0"
I20260929 10:50:18.818557 25782 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:50:18.819325 25796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:18.819514 25782 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:50:18.819600 25782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-2/wal
uuid: "3881f10a1d574f4ba23c983a4f686118"
format_stamp: "Formatted at 2026-09-29 10:50:18 on dist-test-slave-vqch"
server_key: "f7526172778f2207a74b81623905d408"
server_key_iv: "ec96f5b43ce605c08af1fa6361d37bf0"
server_key_version: "encryptionkey@0"
I20260929 10:50:18.819654 25782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:18.822937 25782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:18.823199 25782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:18.823340 25782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:18.823545 25782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:18.823849 25782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:18.823911 25782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:18.823949 25782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:18.823975 25782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:18.829803 25782 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.195:43625
I20260929 10:50:18.829908 25910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.195:43625 every 8 connection(s)
I20260929 10:50:18.830224 25782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
I20260929 10:50:18.833498 25497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:18.831257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:44725 (local address 127.20.80.254:38235)
0929 10:50:18.831315 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:18.831317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:18.831583 (+ 266us) server_negotiation.cc:407] Connection header received
0929 10:50:18.832042 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:18.832043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:18.832049 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:18.832099 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:18.833284 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:18.833286 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:18.833286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:50:18.833290 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:18.833306 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:50:18.833311 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:18.833369 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:18.833434 (+ 65us) server_negotiation.cc:299] Negotiation successful
0929 10:50:18.833445 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 10:50:18.833690 25913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:18.831257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:38235 (local address 127.20.80.195:44725)
0929 10:50:18.831459 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:18.831492 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:18.831950 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:18.832160 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:18.832163 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:18.832172 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:18.833141 (+ 969us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:50:18.833142 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:18.833169 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:50:18.833198 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:18.833371 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:18.833373 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:18.833376 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:50:18.833427 (+ 51us) client_negotiation.cc:240] Negotiation successful
0929 10:50:18.833473 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":102,"threads_started":1}
I20260929 10:50:18.834254 25911 heartbeater.cc:347] Connected to a master server at 127.20.80.254:38235
I20260929 10:50:18.834358 25911 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:18.834523 25911 heartbeater.cc:511] Master 127.20.80.254:38235 requested a full tablet report, sending...
I20260929 10:50:18.834921 25457 ts_manager.cc:194] Registered new tserver with Master: 3881f10a1d574f4ba23c983a4f686118 (127.20.80.195:43625)
I20260929 10:50:18.838544 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 25782
I20260929 10:50:18.838657 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance
I20260929 10:50:18.838840 20803 external_mini_cluster.cc:1428] Setting key dd784b585da5082d8d61ab48132ffe22
I20260929 10:50:18.839792 20803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:50:18.842466 25497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:18.841357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57370 (local address 127.20.80.254:38235)
0929 10:50:18.841441 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:18.841443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:18.841570 (+ 127us) server_negotiation.cc:407] Connection header received
0929 10:50:18.841611 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:18.841612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:18.841617 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:18.841668 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:18.842200 (+ 532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:18.842201 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:18.842202 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:50:18.842206 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:18.842220 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:50:18.842223 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:18.842280 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:18.842337 (+ 57us) server_negotiation.cc:299] Negotiation successful
0929 10:50:18.842373 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 10:50:18.844262 25457 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57370:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:50:18.845906 25457 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:50:18.849678 25921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:18.847950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:39881 (local address 127.0.0.1:48762)
0929 10:50:18.848298 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:18.848340 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:18.848397 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:18.848670 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:18.848676 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:18.848683 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:18.849241 (+ 558us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:50:18.849242 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:18.849260 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:50:18.849286 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:18.849525 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:18.849531 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:18.849534 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:50:18.849598 (+ 64us) client_negotiation.cc:240] Negotiation successful
0929 10:50:18.849613 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":78,"threads_started":1}
I20260929 10:50:18.849705 25922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:18.848045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48762 (local address 127.20.80.193:39881)
0929 10:50:18.848519 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:18.848523 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:18.848532 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:50:18.848557 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:18.848559 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:18.848568 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:18.848676 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:18.849355 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:18.849363 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:18.849363 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:50:18.849367 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:18.849404 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:50:18.849431 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:18.849497 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:18.849608 (+ 111us) server_negotiation.cc:299] Negotiation successful
0929 10:50:18.849624 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":70,"threads_started":1}
I20260929 10:50:18.849987 25925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:18.848247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53392 (local address 127.20.80.194:36005)
0929 10:50:18.848416 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:18.848422 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:18.848501 (+ 79us) server_negotiation.cc:407] Connection header received
0929 10:50:18.848570 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:18.848573 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:18.848584 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:18.848662 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:18.849484 (+ 822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:18.849492 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:18.849493 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:50:18.849497 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:18.849532 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:50:18.849556 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:18.849612 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:18.849860 (+ 248us) server_negotiation.cc:299] Negotiation successful
0929 10:50:18.849903 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":96,"threads_started":1}
I20260929 10:50:18.849797 25924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:18.848166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:36005 (local address 127.0.0.1:53392)
0929 10:50:18.848446 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:18.848456 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:18.848508 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:18.849025 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:18.849026 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:18.849029 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:18.849400 (+ 371us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:50:18.849400 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:18.849405 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:50:18.849414 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:18.849640 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:18.849641 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:18.849641 (+ 0us) client_negotiation.cc:769] Sending connection context
0929 10:50:18.849700 (+ 59us) client_negotiation.cc:240] Negotiation successful
0929 10:50:18.849714 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":61,"threads_started":1}
I20260929 10:50:18.850242 25920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:18.847993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:43625 (local address 127.0.0.1:40450)
0929 10:50:18.848299 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:18.848340 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:18.848397 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:18.849393 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:18.849394 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:18.849397 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:18.849824 (+ 427us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:50:18.849825 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:18.849833 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:50:18.849847 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:18.850097 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:18.850098 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:18.850099 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 10:50:18.850147 (+ 48us) client_negotiation.cc:240] Negotiation successful
0929 10:50:18.850178 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":102,"threads_started":1}
I20260929 10:50:18.850512 25923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:18.848102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40450 (local address 127.20.80.195:43625)
0929 10:50:18.848638 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:18.848642 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:18.848651 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:50:18.848973 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:18.849006 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:18.849017 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:18.849092 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:18.849931 (+ 839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:18.849938 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:18.849939 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:50:18.849943 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:18.849976 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:50:18.850003 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:18.850067 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:18.850409 (+ 342us) server_negotiation.cc:299] Negotiation successful
0929 10:50:18.850428 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":112,"threads_started":1}
I20260929 10:50:18.850807 25711 tablet_service.cc:1467] Processing CreateTablet for tablet dc6763b85f094690981d3a2b73111209 (DEFAULT_TABLE table=test-table [id=8e40c2a93d8e4ae1892ab470bd6dc840]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:18.850879 25580 tablet_service.cc:1467] Processing CreateTablet for tablet dc6763b85f094690981d3a2b73111209 (DEFAULT_TABLE table=test-table [id=8e40c2a93d8e4ae1892ab470bd6dc840]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:18.851198 25580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc6763b85f094690981d3a2b73111209. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:18.851198 25711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc6763b85f094690981d3a2b73111209. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:18.851116 25845 tablet_service.cc:1467] Processing CreateTablet for tablet dc6763b85f094690981d3a2b73111209 (DEFAULT_TABLE table=test-table [id=8e40c2a93d8e4ae1892ab470bd6dc840]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:18.851438 25845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc6763b85f094690981d3a2b73111209. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:18.854151 25926 tablet_bootstrap.cc:492] T dc6763b85f094690981d3a2b73111209 P 3881f10a1d574f4ba23c983a4f686118: Bootstrap starting.
I20260929 10:50:18.854439 25928 tablet_bootstrap.cc:492] T dc6763b85f094690981d3a2b73111209 P f9a98ddaa95142199c072d4b5b287bd6: Bootstrap starting.
I20260929 10:50:18.855101 25928 tablet_bootstrap.cc:654] T dc6763b85f094690981d3a2b73111209 P f9a98ddaa95142199c072d4b5b287bd6: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:18.855101 25926 tablet_bootstrap.cc:654] T dc6763b85f094690981d3a2b73111209 P 3881f10a1d574f4ba23c983a4f686118: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:18.855441 25928 log.cc:824] T dc6763b85f094690981d3a2b73111209 P f9a98ddaa95142199c072d4b5b287bd6: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:18.855456 25926 log.cc:824] T dc6763b85f094690981d3a2b73111209 P 3881f10a1d574f4ba23c983a4f686118: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:18.855813 25927 tablet_bootstrap.cc:492] T dc6763b85f094690981d3a2b73111209 P 015069fa88dd42f79301cce68f68b787: Bootstrap starting.
I20260929 10:50:18.856274 25928 tablet_bootstrap.cc:492] T dc6763b85f094690981d3a2b73111209 P f9a98ddaa95142199c072d4b5b287bd6: No bootstrap required, opened a new log
I20260929 10:50:18.856357 25927 tablet_bootstrap.cc:654] T dc6763b85f094690981d3a2b73111209 P 015069fa88dd42f79301cce68f68b787: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:18.856359 25928 ts_tablet_manager.cc:1397] T dc6763b85f094690981d3a2b73111209 P f9a98ddaa95142199c072d4b5b287bd6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:50:18.856485 25926 tablet_bootstrap.cc:492] T dc6763b85f094690981d3a2b73111209 P 3881f10a1d574f4ba23c983a4f686118: No bootstrap required, opened a new log
I20260929 10:50:18.856559 25926 ts_tablet_manager.cc:1397] T dc6763b85f094690981d3a2b73111209 P 3881f10a1d574f4ba23c983a4f686118: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:50:18.856695 25927 log.cc:824] T dc6763b85f094690981d3a2b73111209 P 015069fa88dd42f79301cce68f68b787: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:18.857476 25927 tablet_bootstrap.cc:492] T dc6763b85f094690981d3a2b73111209 P 015069fa88dd42f79301cce68f68b787: No bootstrap required, opened a new log
I20260929 10:50:18.857545 25927 ts_tablet_manager.cc:1397] T dc6763b85f094690981d3a2b73111209 P 015069fa88dd42f79301cce68f68b787: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:50:18.858728 25928 raft_consensus.cc:348] T dc6763b85f094690981d3a2b73111209 P f9a98ddaa95142199c072d4b5b287bd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a98ddaa95142199c072d4b5b287bd6" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 39881 } } peers { permanent_uuid: "015069fa88dd42f79301cce68f68b787" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 36005 } } peers { permanent_uuid: "3881f10a1d574f4ba23c983a4f686118" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 43625 } }
I20260929 10:50:18.858889 25928 raft_consensus.cc:374] T dc6763b85f094690981d3a2b73111209 P f9a98ddaa95142199c072d4b5b287bd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:18.858935 25928 raft_consensus.cc:729] T dc6763b85f094690981d3a2b73111209 P f9a98ddaa95142199c072d4b5b287bd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9a98ddaa95142199c072d4b5b287bd6, State: Initialized, Role: FOLLOWER
I20260929 10:50:18.859071 25928 consensus_queue.cc:260] T dc6763b85f094690981d3a2b73111209 P f9a98ddaa95142199c072d4b5b287bd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a98ddaa95142199c072d4b5b287bd6" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 39881 } } peers { permanent_uuid: "015069fa88dd42f79301cce68f68b787" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 36005 } } peers { permanent_uuid: "3881f10a1d574f4ba23c983a4f686118" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 43625 } }
I20260929 10:50:18.859328 25928 ts_tablet_manager.cc:1428] T dc6763b85f094690981d3a2b73111209 P f9a98ddaa95142199c072d4b5b287bd6: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260929 10:50:18.859211 25926 raft_consensus.cc:348] T dc6763b85f094690981d3a2b73111209 P 3881f10a1d574f4ba23c983a4f686118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a98ddaa95142199c072d4b5b287bd6" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 39881 } } peers { permanent_uuid: "015069fa88dd42f79301cce68f68b787" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 36005 } } peers { permanent_uuid: "3881f10a1d574f4ba23c983a4f686118" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 43625 } }
I20260929 10:50:18.859387 25926 raft_consensus.cc:374] T dc6763b85f094690981d3a2b73111209 P 3881f10a1d574f4ba23c983a4f686118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:18.859483 25646 heartbeater.cc:503] Master 127.20.80.254:38235 was elected leader, sending a full tablet report...
I20260929 10:50:18.859758 25926 raft_consensus.cc:729] T dc6763b85f094690981d3a2b73111209 P 3881f10a1d574f4ba23c983a4f686118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3881f10a1d574f4ba23c983a4f686118, State: Initialized, Role: FOLLOWER
I20260929 10:50:18.859926 25926 consensus_queue.cc:260] T dc6763b85f094690981d3a2b73111209 P 3881f10a1d574f4ba23c983a4f686118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a98ddaa95142199c072d4b5b287bd6" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 39881 } } peers { permanent_uuid: "015069fa88dd42f79301cce68f68b787" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 36005 } } peers { permanent_uuid: "3881f10a1d574f4ba23c983a4f686118" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 43625 } }
I20260929 10:50:18.860193 25926 ts_tablet_manager.cc:1428] T dc6763b85f094690981d3a2b73111209 P 3881f10a1d574f4ba23c983a4f686118: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260929 10:50:18.860209 25911 heartbeater.cc:503] Master 127.20.80.254:38235 was elected leader, sending a full tablet report...
I20260929 10:50:18.860170 25927 raft_consensus.cc:348] T dc6763b85f094690981d3a2b73111209 P 015069fa88dd42f79301cce68f68b787 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a98ddaa95142199c072d4b5b287bd6" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 39881 } } peers { permanent_uuid: "015069fa88dd42f79301cce68f68b787" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 36005 } } peers { permanent_uuid: "3881f10a1d574f4ba23c983a4f686118" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 43625 } }
I20260929 10:50:18.860317 25927 raft_consensus.cc:374] T dc6763b85f094690981d3a2b73111209 P 015069fa88dd42f79301cce68f68b787 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:18.860351 25927 raft_consensus.cc:729] T dc6763b85f094690981d3a2b73111209 P 015069fa88dd42f79301cce68f68b787 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015069fa88dd42f79301cce68f68b787, State: Initialized, Role: FOLLOWER
I20260929 10:50:18.860445 25927 consensus_queue.cc:260] T dc6763b85f094690981d3a2b73111209 P 015069fa88dd42f79301cce68f68b787 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a98ddaa95142199c072d4b5b287bd6" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 39881 } } peers { permanent_uuid: "015069fa88dd42f79301cce68f68b787" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 36005 } } peers { permanent_uuid: "3881f10a1d574f4ba23c983a4f686118" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 43625 } }
I20260929 10:50:18.860718 25927 ts_tablet_manager.cc:1428] T dc6763b85f094690981d3a2b73111209 P 015069fa88dd42f79301cce68f68b787: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 10:50:18.860739 25779 heartbeater.cc:503] Master 127.20.80.254:38235 was elected leader, sending a full tablet report...
I20260929 10:50:18.891480 25932 raft_consensus.cc:482] T dc6763b85f094690981d3a2b73111209 P f9a98ddaa95142199c072d4b5b287bd6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:50:18.891592 25932 raft_consensus.cc:504] T dc6763b85f094690981d3a2b73111209 P f9a98ddaa95142199c072d4b5b287bd6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a98ddaa95142199c072d4b5b287bd6" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 39881 } } peers { permanent_uuid: "015069fa88dd42f79301cce68f68b787" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 36005 } } peers { permanent_uuid: "3881f10a1d574f4ba23c983a4f686118" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 43625 } }
I20260929 10:50:18.891916 25932 leader_election.cc:288] T dc6763b85f094690981d3a2b73111209 P f9a98ddaa95142199c072d4b5b287bd6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015069fa88dd42f79301cce68f68b787 (127.20.80.194:36005), 3881f10a1d574f4ba23c983a4f686118 (127.20.80.195:43625)
I20260929 10:50:18.893277 25648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:18.892081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:43625 (local address 127.20.80.193:41517)
0929 10:50:18.892133 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:18.892147 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:18.892196 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:18.892367 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:18.892368 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:18.892371 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:18.892933 (+ 562us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:50:18.892934 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:18.892942 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:50:18.892957 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:18.893172 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:18.893173 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:18.893174 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 10:50:18.893217 (+ 43us) client_negotiation.cc:240] Negotiation successful
0929 10:50:18.893230 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260929 10:50:18.893452 25923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:18.892170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:41517 (local address 127.20.80.195:43625)
0929 10:50:18.892222 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:18.892225 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:18.892237 (+ 12us) server_negotiation.cc:407] Connection header received
0929 10:50:18.892266 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:18.892267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:18.892273 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:18.892362 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:18.893071 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:18.893073 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:18.893073 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:50:18.893078 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:18.893097 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:50:18.893102 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:18.893168 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:18.893359 (+ 191us) server_negotiation.cc:299] Negotiation successful
0929 10:50:18.893374 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260929 10:50:18.893530 25925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:18.892201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:50293 (local address 127.20.80.194:36005)
0929 10:50:18.892254 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:18.892256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:18.892377 (+ 121us) server_negotiation.cc:407] Connection header received
0929 10:50:18.892511 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:18.892512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:18.892518 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:18.892592 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:18.893247 (+ 655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:18.893248 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:18.893249 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:50:18.893253 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:18.893269 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:50:18.893273 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:18.893330 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:18.893424 (+ 94us) server_negotiation.cc:299] Negotiation successful
0929 10:50:18.893438 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260929 10:50:18.893700 25935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:18.892083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:36005 (local address 127.20.80.193:50293)
0929 10:50:18.892313 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:18.892320 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:18.892377 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:18.892645 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:18.892646 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:18.892648 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:18.893154 (+ 506us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:50:18.893155 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:18.893162 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:50:18.893174 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:18.893332 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:18.893333 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:18.893334 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 10:50:18.893603 (+ 269us) client_negotiation.cc:240] Negotiation successful
0929 10:50:18.893614 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":79,"thread_start_us":93,"threads_started":1}
I20260929 10:50:18.893798 25865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc6763b85f094690981d3a2b73111209" candidate_uuid: "f9a98ddaa95142199c072d4b5b287bd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3881f10a1d574f4ba23c983a4f686118" is_pre_election: true
I20260929 10:50:18.893965 25865 raft_consensus.cc:2454] T dc6763b85f094690981d3a2b73111209 P 3881f10a1d574f4ba23c983a4f686118 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9a98ddaa95142199c072d4b5b287bd6 in term 0.
I20260929 10:50:18.894111 25731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc6763b85f094690981d3a2b73111209" candidate_uuid: "f9a98ddaa95142199c072d4b5b287bd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015069fa88dd42f79301cce68f68b787" is_pre_election: true
I20260929 10:50:18.894179 25536 leader_election.cc:302] T dc6763b85f094690981d3a2b73111209 P f9a98ddaa95142199c072d4b5b287bd6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3881f10a1d574f4ba23c983a4f686118, f9a98ddaa95142199c072d4b5b287bd6; no voters:
I20260929 10:50:18.894277 25731 raft_consensus.cc:2454] T dc6763b85f094690981d3a2b73111209 P 015069fa88dd42f79301cce68f68b787 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9a98ddaa95142199c072d4b5b287bd6 in term 0.
I20260929 10:50:18.894289 25932 raft_consensus.cc:2781] T dc6763b85f094690981d3a2b73111209 P f9a98ddaa95142199c072d4b5b287bd6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:50:18.894335 25932 raft_consensus.cc:482] T dc6763b85f094690981d3a2b73111209 P f9a98ddaa95142199c072d4b5b287bd6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:50:18.894375 25932 raft_consensus.cc:3037] T dc6763b85f094690981d3a2b73111209 P f9a98ddaa95142199c072d4b5b287bd6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:18.924098 25932 raft_consensus.cc:504] T dc6763b85f094690981d3a2b73111209 P f9a98ddaa95142199c072d4b5b287bd6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a98ddaa95142199c072d4b5b287bd6" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 39881 } } peers { permanent_uuid: "015069fa88dd42f79301cce68f68b787" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 36005 } } peers { permanent_uuid: "3881f10a1d574f4ba23c983a4f686118" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 43625 } }
I20260929 10:50:18.924325 25932 leader_election.cc:288] T dc6763b85f094690981d3a2b73111209 P f9a98ddaa95142199c072d4b5b287bd6 [CANDIDATE]: Term 1 election: Requested vote from peers 015069fa88dd42f79301cce68f68b787 (127.20.80.194:36005), 3881f10a1d574f4ba23c983a4f686118 (127.20.80.195:43625)
I20260929 10:50:18.924530 25731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc6763b85f094690981d3a2b73111209" candidate_uuid: "f9a98ddaa95142199c072d4b5b287bd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015069fa88dd42f79301cce68f68b787"
I20260929 10:50:18.924592 25865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc6763b85f094690981d3a2b73111209" candidate_uuid: "f9a98ddaa95142199c072d4b5b287bd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3881f10a1d574f4ba23c983a4f686118"
I20260929 10:50:18.924679 25865 raft_consensus.cc:3037] T dc6763b85f094690981d3a2b73111209 P 3881f10a1d574f4ba23c983a4f686118 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:18.924664 25731 raft_consensus.cc:3037] T dc6763b85f094690981d3a2b73111209 P 015069fa88dd42f79301cce68f68b787 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:18.925479 25731 raft_consensus.cc:2454] T dc6763b85f094690981d3a2b73111209 P 015069fa88dd42f79301cce68f68b787 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9a98ddaa95142199c072d4b5b287bd6 in term 1.
I20260929 10:50:18.925688 25534 leader_election.cc:302] T dc6763b85f094690981d3a2b73111209 P f9a98ddaa95142199c072d4b5b287bd6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015069fa88dd42f79301cce68f68b787, f9a98ddaa95142199c072d4b5b287bd6; no voters:
I20260929 10:50:18.925799 25932 raft_consensus.cc:2781] T dc6763b85f094690981d3a2b73111209 P f9a98ddaa95142199c072d4b5b287bd6 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:18.925796 25865 raft_consensus.cc:2454] T dc6763b85f094690981d3a2b73111209 P 3881f10a1d574f4ba23c983a4f686118 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9a98ddaa95142199c072d4b5b287bd6 in term 1.
I20260929 10:50:18.925956 25932 raft_consensus.cc:686] T dc6763b85f094690981d3a2b73111209 P f9a98ddaa95142199c072d4b5b287bd6 [term 1 LEADER]: Becoming Leader. State: Replica: f9a98ddaa95142199c072d4b5b287bd6, State: Running, Role: LEADER
I20260929 10:50:18.926061 25932 consensus_queue.cc:237] T dc6763b85f094690981d3a2b73111209 P f9a98ddaa95142199c072d4b5b287bd6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a98ddaa95142199c072d4b5b287bd6" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 39881 } } peers { permanent_uuid: "015069fa88dd42f79301cce68f68b787" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 36005 } } peers { permanent_uuid: "3881f10a1d574f4ba23c983a4f686118" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 43625 } }
I20260929 10:50:18.926957 25457 catalog_manager.cc:5495] T dc6763b85f094690981d3a2b73111209 P f9a98ddaa95142199c072d4b5b287bd6 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9a98ddaa95142199c072d4b5b287bd6 (127.20.80.193). New cstate: current_term: 1 leader_uuid: "f9a98ddaa95142199c072d4b5b287bd6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a98ddaa95142199c072d4b5b287bd6" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 39881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015069fa88dd42f79301cce68f68b787" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 36005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3881f10a1d574f4ba23c983a4f686118" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 43625 } health_report { overall_health: UNKNOWN } } }
I20260929 10:50:18.944130 25922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:18.943083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48774 (local address 127.20.80.193:39881)
0929 10:50:18.943141 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:18.943143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:18.943164 (+ 21us) server_negotiation.cc:407] Connection header received
0929 10:50:18.943202 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:18.943203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:18.943217 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:18.943280 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:18.943910 (+ 630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:18.943912 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:18.943912 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:50:18.943917 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:18.943932 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:50:18.943937 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:18.943982 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:18.944041 (+ 59us) server_negotiation.cc:299] Negotiation successful
0929 10:50:18.944055 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260929 10:50:18.945895 25865 raft_consensus.cc:1261] T dc6763b85f094690981d3a2b73111209 P 3881f10a1d574f4ba23c983a4f686118 [term 1 FOLLOWER]: Refusing update from remote peer f9a98ddaa95142199c072d4b5b287bd6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:18.945940 25731 raft_consensus.cc:1261] T dc6763b85f094690981d3a2b73111209 P 015069fa88dd42f79301cce68f68b787 [term 1 FOLLOWER]: Refusing update from remote peer f9a98ddaa95142199c072d4b5b287bd6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:18.946254 25936 consensus_queue.cc:1035] T dc6763b85f094690981d3a2b73111209 P f9a98ddaa95142199c072d4b5b287bd6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015069fa88dd42f79301cce68f68b787" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 36005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:50:18.946372 25932 consensus_queue.cc:1035] T dc6763b85f094690981d3a2b73111209 P f9a98ddaa95142199c072d4b5b287bd6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3881f10a1d574f4ba23c983a4f686118" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 43625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:50:18.949015 25940 mvcc.cc:205] Tried to move back new op lower bound from 7334621261599870976 to 7334621261521756160. Current Snapshot: MvccSnapshot[applied={T|T < 7334621261599870976}]
I20260929 10:50:18.949275 25939 mvcc.cc:205] Tried to move back new op lower bound from 7334621261599870976 to 7334621261521756160. Current Snapshot: MvccSnapshot[applied={T|T < 7334621261599870976 or (T in {7334621261599870976})}]
I20260929 10:50:18.949395 25941 mvcc.cc:205] Tried to move back new op lower bound from 7334621261599870976 to 7334621261521756160. Current Snapshot: MvccSnapshot[applied={T|T < 7334621261599870976}]
I20260929 10:50:18.950831 25456 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:50:18.950950 25456 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:50:18.951675 25456 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 261c150f414d4f2c9d00d8615d75f02e: Sending DeleteTablet for 3 replicas of tablet dc6763b85f094690981d3a2b73111209
I20260929 10:50:18.951992 25580 tablet_service.cc:1514] Processing DeleteTablet for tablet dc6763b85f094690981d3a2b73111209 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:18 UTC) from {username='slave'} at 127.0.0.1:48762
I20260929 10:50:18.952044 25845 tablet_service.cc:1514] Processing DeleteTablet for tablet dc6763b85f094690981d3a2b73111209 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:18 UTC) from {username='slave'} at 127.0.0.1:40450
I20260929 10:50:18.952054 25711 tablet_service.cc:1514] Processing DeleteTablet for tablet dc6763b85f094690981d3a2b73111209 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:18 UTC) from {username='slave'} at 127.0.0.1:53392
I20260929 10:50:18.952201 25949 tablet_replica.cc:330] T dc6763b85f094690981d3a2b73111209 P f9a98ddaa95142199c072d4b5b287bd6: stopping tablet replica
I20260929 10:50:18.952294 25949 raft_consensus.cc:2229] T dc6763b85f094690981d3a2b73111209 P f9a98ddaa95142199c072d4b5b287bd6 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:50:18.952455 25949 raft_consensus.cc:2258] T dc6763b85f094690981d3a2b73111209 P f9a98ddaa95142199c072d4b5b287bd6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:18.952450 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 25518
I20260929 10:50:18.954051 25950 tablet_replica.cc:330] T dc6763b85f094690981d3a2b73111209 P 3881f10a1d574f4ba23c983a4f686118: stopping tablet replica
I20260929 10:50:18.954166 25950 raft_consensus.cc:2229] T dc6763b85f094690981d3a2b73111209 P 3881f10a1d574f4ba23c983a4f686118 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:50:18.954267 25950 raft_consensus.cc:2258] T dc6763b85f094690981d3a2b73111209 P 3881f10a1d574f4ba23c983a4f686118 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:18.954430 25951 tablet_replica.cc:330] T dc6763b85f094690981d3a2b73111209 P 015069fa88dd42f79301cce68f68b787: stopping tablet replica
I20260929 10:50:18.954522 25951 raft_consensus.cc:2229] T dc6763b85f094690981d3a2b73111209 P 015069fa88dd42f79301cce68f68b787 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:50:18.954613 25951 raft_consensus.cc:2258] T dc6763b85f094690981d3a2b73111209 P 015069fa88dd42f79301cce68f68b787 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:18.954746 25950 ts_tablet_manager.cc:1905] T dc6763b85f094690981d3a2b73111209 P 3881f10a1d574f4ba23c983a4f686118: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:18.955003 25951 ts_tablet_manager.cc:1905] T dc6763b85f094690981d3a2b73111209 P 015069fa88dd42f79301cce68f68b787: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260929 10:50:18.959039 25442 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.80.193:39881 (error 108)
I20260929 10:50:18.959178 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 25649
I20260929 10:50:18.961566 25920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:18.961438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:39881 (local address 127.0.0.1:48788)
0929 10:50:18.961493 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:18.961536 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.193:39881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260929 10:50:18.961706 25442 catalog_manager.cc:4584] TS f9a98ddaa95142199c072d4b5b287bd6 (127.20.80.193:39881): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dc6763b85f094690981d3a2b73111209: Network error: Client connection negotiation failed: client connection to 127.20.80.193:39881: connect: Connection refused (error 111)
I20260929 10:50:19.014575 25920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:19.014421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:39881 (local address 127.0.0.1:48798)
0929 10:50:19.014486 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:19.014521 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.193:39881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260929 10:50:19.014968 25920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:19.014872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:39881 (local address 127.0.0.1:48804)
0929 10:50:19.014916 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:19.014940 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.193:39881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260929 10:50:19.025751 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 25782
I20260929 10:50:19.025897 25920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:19.025751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:36005 (local address 127.0.0.1:53404)
0929 10:50:19.025803 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:19.025828 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.194:36005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260929 10:50:19.031672 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 25426
I20260929 10:50:19.031802 25920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:19.031665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:43625 (local address 127.0.0.1:40462)
0929 10:50:19.031729 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:19.031759 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.195:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
2026-09-29T10:50:19Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (813 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260929 10:50:19.101742 20803 mini_oidc.cc:152] Starting JWKS server
I20260929 10:50:19.102828 20803 webserver.cc:460] Webserver started at https://127.0.0.1:40369/ using document root <none> and password file <none>
I20260929 10:50:19.102946 20803 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 10:50:19.103144 20803 webserver.cc:460] Webserver started at http://127.0.0.1:36745/ using document root <none> and password file <none>
2026-09-29T10:50:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:50:19Z Disabled control of system clock
I20260929 10:50:19.110945 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:46047
--webserver_interface=127.20.80.254
--webserver_port=0
--builtin_ntp_servers=127.20.80.212:41419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:46047
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40369/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790678996126930-20803-0/testchainca.cert with env {}
W20260929 10:50:19.192184 25958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:19.192384 25958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:19.192425 25958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:19.193682 25958 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:50:19.193722 25958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:19.193735 25958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:19.193748 25958 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:50:19.193790 25958 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 10:50:19.193809 25958 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 10:50:19.193830 25958 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40369/jwks.json
I20260929 10:50:19.195458 25958 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:41419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:46047
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:46047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40369/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790678996126930-20803-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:19.195700 25958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:19.195931 25958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:19.200770 25963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:19.201033 25964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:19.201017 25966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:19.201778 25958 server_base.cc:1034] running on GCE node
I20260929 10:50:19.202020 25958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:19.202308 25958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:19.203469 25958 hybrid_clock.cc:648] HybridClock initialized: now 1790679019203455 us; error 27 us; skew 500 ppm
I20260929 10:50:19.204803 25958 webserver.cc:460] Webserver started at http://127.20.80.254:43469/ using document root <none> and password file <none>
I20260929 10:50:19.205119 25958 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:19.205168 25958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:19.205312 25958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:19.206373 25958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "e41e1f8bbb5242cd8c9912b5afb9bd25"
format_stamp: "Formatted at 2026-09-29 10:50:19 on dist-test-slave-vqch"
server_key: "3dd4981fd7941f3d2ae5cf905eb89e31"
server_key_iv: "dff3f4b7a6d8263528b53acee69fa515"
server_key_version: "encryptionkey@0"
I20260929 10:50:19.206746 25958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "e41e1f8bbb5242cd8c9912b5afb9bd25"
format_stamp: "Formatted at 2026-09-29 10:50:19 on dist-test-slave-vqch"
server_key: "3dd4981fd7941f3d2ae5cf905eb89e31"
server_key_iv: "dff3f4b7a6d8263528b53acee69fa515"
server_key_version: "encryptionkey@0"
I20260929 10:50:19.208011 25958 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:50:19.208729 25972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:19.208905 25958 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:19.209023 25958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "e41e1f8bbb5242cd8c9912b5afb9bd25"
format_stamp: "Formatted at 2026-09-29 10:50:19 on dist-test-slave-vqch"
server_key: "3dd4981fd7941f3d2ae5cf905eb89e31"
server_key_iv: "dff3f4b7a6d8263528b53acee69fa515"
server_key_version: "encryptionkey@0"
I20260929 10:50:19.209110 25958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:19.227833 25958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:19.228196 25958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:19.228367 25958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:19.232853 25958 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:46047
I20260929 10:50:19.233321 25958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:50:19.232893 26024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:46047 every 8 connection(s)
I20260929 10:50:19.234320 26025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:19.235221 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 25958
I20260929 10:50:19.235366 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:50:19.235540 20803 external_mini_cluster.cc:1428] Setting key 17feb235fdbe351700cfe5ba7492b41b
I20260929 10:50:19.236909 26025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e41e1f8bbb5242cd8c9912b5afb9bd25: Bootstrap starting.
I20260929 10:50:19.237579 26025 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e41e1f8bbb5242cd8c9912b5afb9bd25: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:19.237883 26025 log.cc:824] T 00000000000000000000000000000000 P e41e1f8bbb5242cd8c9912b5afb9bd25: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:19.238569 26025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e41e1f8bbb5242cd8c9912b5afb9bd25: No bootstrap required, opened a new log
I20260929 10:50:19.240595 26025 raft_consensus.cc:348] T 00000000000000000000000000000000 P e41e1f8bbb5242cd8c9912b5afb9bd25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41e1f8bbb5242cd8c9912b5afb9bd25" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46047 } }
I20260929 10:50:19.240770 26025 raft_consensus.cc:374] T 00000000000000000000000000000000 P e41e1f8bbb5242cd8c9912b5afb9bd25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:19.240835 26025 raft_consensus.cc:729] T 00000000000000000000000000000000 P e41e1f8bbb5242cd8c9912b5afb9bd25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e41e1f8bbb5242cd8c9912b5afb9bd25, State: Initialized, Role: FOLLOWER
I20260929 10:50:19.241048 26028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:19.236081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57538 (local address 127.20.80.254:46047)
0929 10:50:19.236266 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:19.236274 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:19.236299 (+ 25us) server_negotiation.cc:407] Connection header received
0929 10:50:19.236715 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:19.236722 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:19.236782 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:19.236899 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:19.237152 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:19.238118 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:19.238969 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:19.239379 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:19.239516 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:19.240233 (+ 717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:19.240243 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:19.240243 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:50:19.240409 (+ 166us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:19.240440 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:50:19.240450 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:19.240522 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:19.240621 (+ 99us) server_negotiation.cc:299] Negotiation successful
0929 10:50:19.240702 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":77,"threads_started":1}
I20260929 10:50:19.241031 26025 consensus_queue.cc:260] T 00000000000000000000000000000000 P e41e1f8bbb5242cd8c9912b5afb9bd25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41e1f8bbb5242cd8c9912b5afb9bd25" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46047 } }
I20260929 10:50:19.241148 26025 raft_consensus.cc:388] T 00000000000000000000000000000000 P e41e1f8bbb5242cd8c9912b5afb9bd25 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:50:19.241189 26025 raft_consensus.cc:482] T 00000000000000000000000000000000 P e41e1f8bbb5242cd8c9912b5afb9bd25 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:50:19.241237 26025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e41e1f8bbb5242cd8c9912b5afb9bd25 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:19.242091 20803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:50:19.321955 26025 raft_consensus.cc:504] T 00000000000000000000000000000000 P e41e1f8bbb5242cd8c9912b5afb9bd25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41e1f8bbb5242cd8c9912b5afb9bd25" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46047 } }
I20260929 10:50:19.322137 26025 leader_election.cc:302] T 00000000000000000000000000000000 P e41e1f8bbb5242cd8c9912b5afb9bd25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e41e1f8bbb5242cd8c9912b5afb9bd25; no voters:
I20260929 10:50:19.322422 26025 leader_election.cc:288] T 00000000000000000000000000000000 P e41e1f8bbb5242cd8c9912b5afb9bd25 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:50:19.322502 26030 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e41e1f8bbb5242cd8c9912b5afb9bd25 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:19.322719 26030 raft_consensus.cc:686] T 00000000000000000000000000000000 P e41e1f8bbb5242cd8c9912b5afb9bd25 [term 1 LEADER]: Becoming Leader. State: Replica: e41e1f8bbb5242cd8c9912b5afb9bd25, State: Running, Role: LEADER
I20260929 10:50:19.322767 26025 sys_catalog.cc:564] T 00000000000000000000000000000000 P e41e1f8bbb5242cd8c9912b5afb9bd25 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:50:19.322898 26030 consensus_queue.cc:237] T 00000000000000000000000000000000 P e41e1f8bbb5242cd8c9912b5afb9bd25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41e1f8bbb5242cd8c9912b5afb9bd25" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46047 } }
I20260929 10:50:19.323354 26031 sys_catalog.cc:455] T 00000000000000000000000000000000 P e41e1f8bbb5242cd8c9912b5afb9bd25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e41e1f8bbb5242cd8c9912b5afb9bd25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41e1f8bbb5242cd8c9912b5afb9bd25" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46047 } } }
I20260929 10:50:19.323451 26031 sys_catalog.cc:458] T 00000000000000000000000000000000 P e41e1f8bbb5242cd8c9912b5afb9bd25 [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:19.323678 26032 sys_catalog.cc:455] T 00000000000000000000000000000000 P e41e1f8bbb5242cd8c9912b5afb9bd25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e41e1f8bbb5242cd8c9912b5afb9bd25. Latest consensus state: current_term: 1 leader_uuid: "e41e1f8bbb5242cd8c9912b5afb9bd25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41e1f8bbb5242cd8c9912b5afb9bd25" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46047 } } }
I20260929 10:50:19.324011 26032 sys_catalog.cc:458] T 00000000000000000000000000000000 P e41e1f8bbb5242cd8c9912b5afb9bd25 [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:19.324051 26035 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:50:19.324561 26035 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:50:19.326601 26035 catalog_manager.cc:1347] Generated new cluster ID: 5fde7e48768645b7a7015ab6a01534db
I20260929 10:50:19.326694 26035 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:50:19.340516 26035 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:50:19.341274 26035 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:50:19.346603 26035 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e41e1f8bbb5242cd8c9912b5afb9bd25: Generated new TSK 0
I20260929 10:50:19.346805 26035 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260929 10:50:19.382601 26056 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260929 10:50:19.382722 26056 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260929 10:50:19.382958 26028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:19.367233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57542 (local address 127.20.80.254:46047)
0929 10:50:19.367318 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:19.367320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:19.367444 (+ 124us) server_negotiation.cc:407] Connection header received
0929 10:50:19.367576 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:19.367576 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:19.367587 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:19.367646 (+ 59us) server_negotiation.cc:226] Negotiated authn=JWT
0929 10:50:19.367824 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:19.368345 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:19.369547 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:19.369756 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:19.369822 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:19.370056 (+ 234us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 10:50:19.382724 (+ 12668us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40369/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0929 10:50:19.382891 (+ 167us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57542: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40369/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":57}
W20260929 10:50:19.383020 26028 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57542: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40369/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260929 10:50:19.383312 26054 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:19.367176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:46047 (local address 127.0.0.1:57542)
0929 10:50:19.367367 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:19.367379 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:19.367456 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:19.367675 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:19.367676 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:19.367679 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0929 10:50:19.367748 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:19.367750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:19.368448 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:19.368449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:19.369423 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:19.369426 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:19.369848 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:19.369849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:19.369917 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:19.369938 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 10:50:19.382910 (+ 12972us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40369/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0929 10:50:19.383020 (+ 110us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.80.254:46047: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40369/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":104,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.20.80.254:46047: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40369/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260929 10:50:19.383711 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 25958
2026-09-29T10:50:19Z chronyd exiting
I20260929 10:50:19.420149 20803 test_util.cc:175] -----------------------------------------------
I20260929 10:50:19.420233 20803 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790678996126930-20803-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (319 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260929 10:50:19.420939 20803 mini_oidc.cc:152] Starting JWKS server
I20260929 10:50:19.421648 20803 webserver.cc:460] Webserver started at https://127.0.0.1:36537/ using document root <none> and password file <none>
I20260929 10:50:19.421689 20803 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 10:50:19.421795 20803 webserver.cc:460] Webserver started at http://127.0.0.1:34741/ using document root <none> and password file <none>
2026-09-29T10:50:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:50:19Z Disabled control of system clock
I20260929 10:50:19.429754 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:35599
--webserver_interface=127.20.80.254
--webserver_port=0
--builtin_ntp_servers=127.20.80.212:38181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:35599
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36537/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790678996126930-20803-0/testchainca.cert with env {}
W20260929 10:50:19.510524 26062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:19.510731 26062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:19.510753 26062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:19.511859 26062 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:50:19.511894 26062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:19.511906 26062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:19.511919 26062 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:50:19.511931 26062 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 10:50:19.511943 26062 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 10:50:19.511955 26062 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36537/jwks.json
I20260929 10:50:19.513494 26062 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:38181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:35599
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:35599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36537/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790678996126930-20803-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:19.513687 26062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:19.513859 26062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:19.518652 26070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:19.518693 26068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:19.518786 26067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:19.519145 26062 server_base.cc:1034] running on GCE node
I20260929 10:50:19.519459 26062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:19.519768 26062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:19.520924 26062 hybrid_clock.cc:648] HybridClock initialized: now 1790679019520906 us; error 29 us; skew 500 ppm
I20260929 10:50:19.522199 26062 webserver.cc:460] Webserver started at http://127.20.80.254:33465/ using document root <none> and password file <none>
I20260929 10:50:19.522478 26062 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:19.522529 26062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:19.522619 26062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:19.523633 26062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "42c7849a993a48f3a481cb34a8406080"
format_stamp: "Formatted at 2026-09-29 10:50:19 on dist-test-slave-vqch"
server_key: "76e81f0c68e194974728710e7546a1f4"
server_key_iv: "241a0ef50124f7e9c440f2d0aeb14ffd"
server_key_version: "encryptionkey@0"
I20260929 10:50:19.523993 26062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "42c7849a993a48f3a481cb34a8406080"
format_stamp: "Formatted at 2026-09-29 10:50:19 on dist-test-slave-vqch"
server_key: "76e81f0c68e194974728710e7546a1f4"
server_key_iv: "241a0ef50124f7e9c440f2d0aeb14ffd"
server_key_version: "encryptionkey@0"
I20260929 10:50:19.525331 26062 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 10:50:19.526114 26076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:19.526298 26062 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 10:50:19.526391 26062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "42c7849a993a48f3a481cb34a8406080"
format_stamp: "Formatted at 2026-09-29 10:50:19 on dist-test-slave-vqch"
server_key: "76e81f0c68e194974728710e7546a1f4"
server_key_iv: "241a0ef50124f7e9c440f2d0aeb14ffd"
server_key_version: "encryptionkey@0"
I20260929 10:50:19.526481 26062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:19.537703 26062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:19.538002 26062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:19.538146 26062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:19.542024 26062 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:35599
I20260929 10:50:19.542057 26128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:35599 every 8 connection(s)
I20260929 10:50:19.542392 26062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:50:19.543038 26129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:19.543762 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 26062
I20260929 10:50:19.543875 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:50:19.544056 20803 external_mini_cluster.cc:1428] Setting key 5cc2352642cbbebd6d025b245f6c8bde
I20260929 10:50:19.546109 26129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42c7849a993a48f3a481cb34a8406080: Bootstrap starting.
I20260929 10:50:19.546705 26129 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42c7849a993a48f3a481cb34a8406080: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:19.546985 26129 log.cc:824] T 00000000000000000000000000000000 P 42c7849a993a48f3a481cb34a8406080: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:19.547673 26129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42c7849a993a48f3a481cb34a8406080: No bootstrap required, opened a new log
I20260929 10:50:19.549660 26132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:19.544579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48250 (local address 127.20.80.254:35599)
0929 10:50:19.544790 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:19.544797 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:19.544821 (+ 24us) server_negotiation.cc:407] Connection header received
0929 10:50:19.545295 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:19.545302 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:19.545372 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:19.545521 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:19.545714 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:19.546627 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:19.547669 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:19.548125 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:19.548270 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:19.549028 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:19.549033 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:19.549033 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:50:19.549207 (+ 174us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:19.549238 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:50:19.549249 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:19.549320 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:19.549423 (+ 103us) server_negotiation.cc:299] Negotiation successful
0929 10:50:19.549503 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":76,"threads_started":1}
I20260929 10:50:19.549865 26129 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42c7849a993a48f3a481cb34a8406080 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c7849a993a48f3a481cb34a8406080" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 35599 } }
I20260929 10:50:19.550052 26129 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42c7849a993a48f3a481cb34a8406080 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:19.550091 26129 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42c7849a993a48f3a481cb34a8406080 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42c7849a993a48f3a481cb34a8406080, State: Initialized, Role: FOLLOWER
I20260929 10:50:19.550204 26129 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42c7849a993a48f3a481cb34a8406080 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c7849a993a48f3a481cb34a8406080" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 35599 } }
I20260929 10:50:19.550311 26129 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42c7849a993a48f3a481cb34a8406080 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:50:19.550369 26129 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42c7849a993a48f3a481cb34a8406080 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:50:19.550419 26129 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42c7849a993a48f3a481cb34a8406080 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:19.550693 20803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:50:19.623072 26129 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42c7849a993a48f3a481cb34a8406080 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c7849a993a48f3a481cb34a8406080" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 35599 } }
I20260929 10:50:19.623284 26129 leader_election.cc:302] T 00000000000000000000000000000000 P 42c7849a993a48f3a481cb34a8406080 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42c7849a993a48f3a481cb34a8406080; no voters:
I20260929 10:50:19.623523 26129 leader_election.cc:288] T 00000000000000000000000000000000 P 42c7849a993a48f3a481cb34a8406080 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:50:19.623631 26134 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42c7849a993a48f3a481cb34a8406080 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:19.623821 26134 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42c7849a993a48f3a481cb34a8406080 [term 1 LEADER]: Becoming Leader. State: Replica: 42c7849a993a48f3a481cb34a8406080, State: Running, Role: LEADER
I20260929 10:50:19.623826 26129 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42c7849a993a48f3a481cb34a8406080 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:50:19.623950 26134 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42c7849a993a48f3a481cb34a8406080 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c7849a993a48f3a481cb34a8406080" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 35599 } }
I20260929 10:50:19.624351 26135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42c7849a993a48f3a481cb34a8406080 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42c7849a993a48f3a481cb34a8406080" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c7849a993a48f3a481cb34a8406080" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 35599 } } }
I20260929 10:50:19.624339 26134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42c7849a993a48f3a481cb34a8406080 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42c7849a993a48f3a481cb34a8406080. Latest consensus state: current_term: 1 leader_uuid: "42c7849a993a48f3a481cb34a8406080" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c7849a993a48f3a481cb34a8406080" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 35599 } } }
I20260929 10:50:19.624500 26134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42c7849a993a48f3a481cb34a8406080 [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:19.624467 26135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42c7849a993a48f3a481cb34a8406080 [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:19.624918 26140 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:50:19.625495 26140 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:50:19.627266 26140 catalog_manager.cc:1347] Generated new cluster ID: c777d57332fe4e6385639537f1693ad9
I20260929 10:50:19.627323 26140 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:50:19.641362 26140 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:50:19.641901 26140 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:50:19.646173 26140 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42c7849a993a48f3a481cb34a8406080: Generated new TSK 0
I20260929 10:50:19.646387 26140 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260929 10:50:19.678746 26159 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260929 10:50:19.678864 26159 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260929 10:50:19.679132 26132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:19.656459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48262 (local address 127.20.80.254:35599)
0929 10:50:19.656526 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:19.656528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:19.656629 (+ 101us) server_negotiation.cc:407] Connection header received
0929 10:50:19.656695 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:19.656696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:19.656709 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:19.656780 (+ 71us) server_negotiation.cc:226] Negotiated authn=JWT
0929 10:50:19.656911 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:19.657770 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:19.658608 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:19.658928 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:19.659000 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:19.659195 (+ 195us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 10:50:19.678910 (+ 19715us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36537/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0929 10:50:19.679071 (+ 161us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48262: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36537/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":39}
W20260929 10:50:19.679208 26132 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48262: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36537/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260929 10:50:19.679275 26158 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:19.656375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:35599 (local address 127.0.0.1:48262)
0929 10:50:19.656552 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:19.656562 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:19.656627 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:19.656785 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:19.656786 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:19.656788 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0929 10:50:19.656847 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:19.656849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:19.657875 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:19.657876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:19.658522 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:19.658524 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:19.659028 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:19.659029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:19.659080 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:19.659086 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 10:50:19.679038 (+ 19952us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36537/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0929 10:50:19.679170 (+ 132us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.80.254:35599: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36537/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":105,"threads_started":1}
I20260929 10:50:19.679800 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 26062
2026-09-29T10:50:19Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (296 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260929 10:50:19.717721 20803 mini_oidc.cc:152] Starting JWKS server
I20260929 10:50:19.718429 20803 webserver.cc:460] Webserver started at https://127.0.0.1:37187/ using document root <none> and password file <none>
I20260929 10:50:19.718477 20803 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 10:50:19.718636 20803 webserver.cc:460] Webserver started at http://127.0.0.1:42377/ using document root <none> and password file <none>
2026-09-29T10:50:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:50:19Z Disabled control of system clock
I20260929 10:50:19.726823 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:44855
--webserver_interface=127.20.80.254
--webserver_port=0
--builtin_ntp_servers=127.20.80.212:46607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:44855
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37187/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260929 10:50:19.807329 26165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:19.807583 26165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:19.807619 26165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:19.808804 26165 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:50:19.808876 26165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:19.808898 26165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:19.808928 26165 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:50:19.808951 26165 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 10:50:19.808998 26165 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 10:50:19.809028 26165 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37187/jwks.json
I20260929 10:50:19.810398 26165 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:46607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:44855
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:44855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37187/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26165
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:19.810683 26165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:19.810983 26165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:19.816002 26173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:19.816002 26171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:19.816023 26170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:19.816264 26165 server_base.cc:1034] running on GCE node
I20260929 10:50:19.816627 26165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:19.816956 26165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:19.818157 26165 hybrid_clock.cc:648] HybridClock initialized: now 1790679019818147 us; error 29 us; skew 500 ppm
I20260929 10:50:19.819387 26165 webserver.cc:460] Webserver started at http://127.20.80.254:41563/ using document root <none> and password file <none>
I20260929 10:50:19.819636 26165 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:19.819705 26165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:19.819859 26165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:19.820958 26165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "479a045c488b4e918281bd92162ffa22"
format_stamp: "Formatted at 2026-09-29 10:50:19 on dist-test-slave-vqch"
server_key: "4888a641bdf58e464f995ac4e2869b42"
server_key_iv: "fc80f085ca5e1deab76fa7a6beb9f736"
server_key_version: "encryptionkey@0"
I20260929 10:50:19.821401 26165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "479a045c488b4e918281bd92162ffa22"
format_stamp: "Formatted at 2026-09-29 10:50:19 on dist-test-slave-vqch"
server_key: "4888a641bdf58e464f995ac4e2869b42"
server_key_iv: "fc80f085ca5e1deab76fa7a6beb9f736"
server_key_version: "encryptionkey@0"
I20260929 10:50:19.822726 26165 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:19.823480 26179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:19.823657 26165 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:19.823757 26165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "479a045c488b4e918281bd92162ffa22"
format_stamp: "Formatted at 2026-09-29 10:50:19 on dist-test-slave-vqch"
server_key: "4888a641bdf58e464f995ac4e2869b42"
server_key_iv: "fc80f085ca5e1deab76fa7a6beb9f736"
server_key_version: "encryptionkey@0"
I20260929 10:50:19.823848 26165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:19.852502 26165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:19.852854 26165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:19.853045 26165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:19.857105 26165 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:44855
I20260929 10:50:19.857148 26232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:44855 every 8 connection(s)
I20260929 10:50:19.857509 26165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:50:19.858181 26233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:19.861615 26233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 479a045c488b4e918281bd92162ffa22: Bootstrap starting.
I20260929 10:50:19.862202 26233 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 479a045c488b4e918281bd92162ffa22: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:19.862514 26233 log.cc:824] T 00000000000000000000000000000000 P 479a045c488b4e918281bd92162ffa22: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:19.862565 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 26165
I20260929 10:50:19.862674 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:50:19.862903 20803 external_mini_cluster.cc:1428] Setting key 62a28c6b97dfa46c65b370eec8acb168
I20260929 10:50:19.863358 26233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 479a045c488b4e918281bd92162ffa22: No bootstrap required, opened a new log
I20260929 10:50:19.866182 26233 raft_consensus.cc:348] T 00000000000000000000000000000000 P 479a045c488b4e918281bd92162ffa22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "479a045c488b4e918281bd92162ffa22" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44855 } }
I20260929 10:50:19.866349 26233 raft_consensus.cc:374] T 00000000000000000000000000000000 P 479a045c488b4e918281bd92162ffa22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:19.866380 26233 raft_consensus.cc:729] T 00000000000000000000000000000000 P 479a045c488b4e918281bd92162ffa22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 479a045c488b4e918281bd92162ffa22, State: Initialized, Role: FOLLOWER
I20260929 10:50:19.866487 26233 consensus_queue.cc:260] T 00000000000000000000000000000000 P 479a045c488b4e918281bd92162ffa22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "479a045c488b4e918281bd92162ffa22" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44855 } }
I20260929 10:50:19.866555 26233 raft_consensus.cc:388] T 00000000000000000000000000000000 P 479a045c488b4e918281bd92162ffa22 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:50:19.866590 26233 raft_consensus.cc:482] T 00000000000000000000000000000000 P 479a045c488b4e918281bd92162ffa22 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:50:19.866647 26233 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 479a045c488b4e918281bd92162ffa22 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:19.869143 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:19.863472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47398 (local address 127.20.80.254:44855)
0929 10:50:19.863695 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:19.863704 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:19.863736 (+ 32us) server_negotiation.cc:407] Connection header received
0929 10:50:19.864666 (+ 930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:19.864676 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:19.864756 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:19.864920 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:19.865152 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:19.866552 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:19.867328 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:19.867739 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:19.867829 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:19.868454 (+ 625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:19.868457 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:19.868458 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:50:19.868664 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:19.868708 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:50:19.868719 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:19.868786 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:19.868864 (+ 78us) server_negotiation.cc:299] Negotiation successful
0929 10:50:19.868940 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":86,"threads_started":1}
I20260929 10:50:19.870075 20803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:50:19.911561 26233 raft_consensus.cc:504] T 00000000000000000000000000000000 P 479a045c488b4e918281bd92162ffa22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "479a045c488b4e918281bd92162ffa22" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44855 } }
I20260929 10:50:19.911756 26233 leader_election.cc:302] T 00000000000000000000000000000000 P 479a045c488b4e918281bd92162ffa22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 479a045c488b4e918281bd92162ffa22; no voters:
I20260929 10:50:19.912016 26233 leader_election.cc:288] T 00000000000000000000000000000000 P 479a045c488b4e918281bd92162ffa22 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:50:19.912106 26238 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 479a045c488b4e918281bd92162ffa22 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:19.912294 26238 raft_consensus.cc:686] T 00000000000000000000000000000000 P 479a045c488b4e918281bd92162ffa22 [term 1 LEADER]: Becoming Leader. State: Replica: 479a045c488b4e918281bd92162ffa22, State: Running, Role: LEADER
I20260929 10:50:19.912344 26233 sys_catalog.cc:564] T 00000000000000000000000000000000 P 479a045c488b4e918281bd92162ffa22 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:50:19.912423 26238 consensus_queue.cc:237] T 00000000000000000000000000000000 P 479a045c488b4e918281bd92162ffa22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "479a045c488b4e918281bd92162ffa22" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44855 } }
I20260929 10:50:19.912812 26240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 479a045c488b4e918281bd92162ffa22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 479a045c488b4e918281bd92162ffa22. Latest consensus state: current_term: 1 leader_uuid: "479a045c488b4e918281bd92162ffa22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "479a045c488b4e918281bd92162ffa22" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44855 } } }
I20260929 10:50:19.912807 26239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 479a045c488b4e918281bd92162ffa22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "479a045c488b4e918281bd92162ffa22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "479a045c488b4e918281bd92162ffa22" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44855 } } }
I20260929 10:50:19.912936 26240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 479a045c488b4e918281bd92162ffa22 [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:19.912940 26239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 479a045c488b4e918281bd92162ffa22 [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:19.913275 26246 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:50:19.913794 26246 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:50:19.915424 26246 catalog_manager.cc:1347] Generated new cluster ID: 0225545522e040c79f04bec10baa2cf3
I20260929 10:50:19.915484 26246 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:50:19.928113 26246 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:50:19.928671 26246 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:50:19.932968 26246 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 479a045c488b4e918281bd92162ffa22: Generated new TSK 0
I20260929 10:50:19.933184 26246 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:50:19.953612 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:19.949969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47412 (local address 127.20.80.254:44855)
0929 10:50:19.950020 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:19.950022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:19.950122 (+ 100us) server_negotiation.cc:407] Connection header received
0929 10:50:19.950196 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:19.950197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:19.950207 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:19.950267 (+ 60us) server_negotiation.cc:226] Negotiated authn=JWT
0929 10:50:19.950407 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:19.950900 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:19.952021 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:19.952203 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:19.952253 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:19.952450 (+ 197us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 10:50:19.953402 (+ 952us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37187/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0929 10:50:19.953535 (+ 133us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47412: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37187/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":28}
W20260929 10:50:19.953678 26237 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47412: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37187/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260929 10:50:19.954008 26262 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:19.949876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44855 (local address 127.0.0.1:47412)
0929 10:50:19.950048 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:19.950059 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:19.950125 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:19.950283 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:19.950284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:19.950286 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0929 10:50:19.950348 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:19.950350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:19.950976 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:19.950977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:19.951939 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:19.951942 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:19.952277 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:19.952279 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:19.952342 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:19.952348 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 10:50:19.953501 (+ 1153us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37187/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0929 10:50:19.953603 (+ 102us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.80.254:44855: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37187/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":104,"threads_started":1}
I20260929 10:50:19.955555 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 26165
E20260929 10:50:19.964315 26263 webserver.cc:533] Webserver: sslize failed:
E20260929 10:50:19.964406 26263 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-29T10:50:19Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (275 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-29T10:50:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:50:19Z Disabled control of system clock
I20260929 10:50:20.000061 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:37883
--webserver_interface=127.20.80.254
--webserver_port=0
--builtin_ntp_servers=127.20.80.212:43011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:37883
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790678996126930-20803-0/testchainca.cert with env {}
W20260929 10:50:20.080765 26267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:20.081017 26267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:20.081066 26267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:20.082240 26267 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:50:20.082306 26267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:20.082329 26267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:20.082369 26267 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:50:20.082396 26267 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 10:50:20.082424 26267 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 10:50:20.082453 26267 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260929 10:50:20.083938 26267 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:43011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:37883
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:37883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790678996126930-20803-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:20.084213 26267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:20.084437 26267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:20.089267 26275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:20.089325 26273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:20.089267 26272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:20.089648 26267 server_base.cc:1034] running on GCE node
I20260929 10:50:20.089879 26267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:20.090200 26267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:20.091368 26267 hybrid_clock.cc:648] HybridClock initialized: now 1790679020091364 us; error 37 us; skew 500 ppm
I20260929 10:50:20.092604 26267 webserver.cc:460] Webserver started at http://127.20.80.254:41647/ using document root <none> and password file <none>
I20260929 10:50:20.092860 26267 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:20.092931 26267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:20.093103 26267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:20.094239 26267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "67c99aef39c842b7b9cd6b46aec7b027"
format_stamp: "Formatted at 2026-09-29 10:50:20 on dist-test-slave-vqch"
server_key: "f38c7e9f2661d615e147b60727ee9cf5"
server_key_iv: "3c35ebcfad6ce0136d0938085f92a038"
server_key_version: "encryptionkey@0"
I20260929 10:50:20.094597 26267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "67c99aef39c842b7b9cd6b46aec7b027"
format_stamp: "Formatted at 2026-09-29 10:50:20 on dist-test-slave-vqch"
server_key: "f38c7e9f2661d615e147b60727ee9cf5"
server_key_iv: "3c35ebcfad6ce0136d0938085f92a038"
server_key_version: "encryptionkey@0"
I20260929 10:50:20.095854 26267 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 10:50:20.096547 26281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:20.096706 26267 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:50:20.096802 26267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "67c99aef39c842b7b9cd6b46aec7b027"
format_stamp: "Formatted at 2026-09-29 10:50:20 on dist-test-slave-vqch"
server_key: "f38c7e9f2661d615e147b60727ee9cf5"
server_key_iv: "3c35ebcfad6ce0136d0938085f92a038"
server_key_version: "encryptionkey@0"
I20260929 10:50:20.096887 26267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:20.107566 26267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:20.107853 26267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:20.108006 26267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:20.111898 26267 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:37883
I20260929 10:50:20.111933 26333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:37883 every 8 connection(s)
I20260929 10:50:20.112280 26267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:50:20.113210 26334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:20.113965 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 26267
I20260929 10:50:20.114075 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:50:20.114271 20803 external_mini_cluster.cc:1428] Setting key d9a654b50c4bfc3fcb6d9c2d0dc4b6df
I20260929 10:50:20.115692 26334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67c99aef39c842b7b9cd6b46aec7b027: Bootstrap starting.
I20260929 10:50:20.116374 26334 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67c99aef39c842b7b9cd6b46aec7b027: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:20.116664 26334 log.cc:824] T 00000000000000000000000000000000 P 67c99aef39c842b7b9cd6b46aec7b027: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:20.117414 26334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67c99aef39c842b7b9cd6b46aec7b027: No bootstrap required, opened a new log
I20260929 10:50:20.119778 26334 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67c99aef39c842b7b9cd6b46aec7b027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67c99aef39c842b7b9cd6b46aec7b027" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37883 } }
I20260929 10:50:20.119957 26334 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67c99aef39c842b7b9cd6b46aec7b027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:20.119964 26337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:20.114782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34272 (local address 127.20.80.254:37883)
0929 10:50:20.114983 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:20.114989 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:20.115014 (+ 25us) server_negotiation.cc:407] Connection header received
0929 10:50:20.115559 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:20.115566 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:20.115628 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:20.115741 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:20.115975 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:20.116965 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:20.117819 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:20.118423 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:20.118549 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:20.119209 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:20.119216 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:20.119217 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:50:20.119453 (+ 236us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:20.119502 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:50:20.119517 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:20.119586 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:20.119641 (+ 55us) server_negotiation.cc:299] Negotiation successful
0929 10:50:20.119721 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":89,"threads_started":1}
I20260929 10:50:20.120009 26334 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67c99aef39c842b7b9cd6b46aec7b027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67c99aef39c842b7b9cd6b46aec7b027, State: Initialized, Role: FOLLOWER
I20260929 10:50:20.120344 26334 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67c99aef39c842b7b9cd6b46aec7b027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67c99aef39c842b7b9cd6b46aec7b027" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37883 } }
I20260929 10:50:20.120465 26334 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67c99aef39c842b7b9cd6b46aec7b027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:50:20.120503 26334 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67c99aef39c842b7b9cd6b46aec7b027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:50:20.120558 26334 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67c99aef39c842b7b9cd6b46aec7b027 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:20.120908 20803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:50:20.121270 26334 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67c99aef39c842b7b9cd6b46aec7b027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67c99aef39c842b7b9cd6b46aec7b027" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37883 } }
I20260929 10:50:20.121393 26334 leader_election.cc:302] T 00000000000000000000000000000000 P 67c99aef39c842b7b9cd6b46aec7b027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67c99aef39c842b7b9cd6b46aec7b027; no voters:
I20260929 10:50:20.121556 26334 leader_election.cc:288] T 00000000000000000000000000000000 P 67c99aef39c842b7b9cd6b46aec7b027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:50:20.121603 26340 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67c99aef39c842b7b9cd6b46aec7b027 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:20.121800 26334 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67c99aef39c842b7b9cd6b46aec7b027 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:50:20.121812 26340 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67c99aef39c842b7b9cd6b46aec7b027 [term 1 LEADER]: Becoming Leader. State: Replica: 67c99aef39c842b7b9cd6b46aec7b027, State: Running, Role: LEADER
I20260929 10:50:20.121913 26340 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67c99aef39c842b7b9cd6b46aec7b027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67c99aef39c842b7b9cd6b46aec7b027" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37883 } }
I20260929 10:50:20.122473 26342 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67c99aef39c842b7b9cd6b46aec7b027 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67c99aef39c842b7b9cd6b46aec7b027. Latest consensus state: current_term: 1 leader_uuid: "67c99aef39c842b7b9cd6b46aec7b027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67c99aef39c842b7b9cd6b46aec7b027" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37883 } } }
I20260929 10:50:20.122556 26342 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67c99aef39c842b7b9cd6b46aec7b027 [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:20.122797 26348 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:50:20.122882 26341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67c99aef39c842b7b9cd6b46aec7b027 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67c99aef39c842b7b9cd6b46aec7b027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67c99aef39c842b7b9cd6b46aec7b027" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37883 } } }
I20260929 10:50:20.122953 26341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67c99aef39c842b7b9cd6b46aec7b027 [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:20.123317 26348 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:50:20.124831 26348 catalog_manager.cc:1347] Generated new cluster ID: e30f9b14741247268db0407a662a471e
I20260929 10:50:20.124931 26348 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:50:20.153878 26348 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:50:20.154440 26348 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:50:20.162267 26348 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67c99aef39c842b7b9cd6b46aec7b027: Generated new TSK 0
I20260929 10:50:20.162472 26348 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:50:20.629277 26337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:20.174004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34284 (local address 127.20.80.254:37883)
0929 10:50:20.174060 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:20.174062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:20.174157 (+ 95us) server_negotiation.cc:407] Connection header received
0929 10:50:20.174215 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:20.174216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:20.174233 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:20.174318 (+ 85us) server_negotiation.cc:226] Negotiated authn=JWT
0929 10:50:20.174444 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:20.175204 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:20.175978 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:20.176275 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:20.176345 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:20.176500 (+ 155us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 10:50:20.629047 (+452547us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0929 10:50:20.629214 (+ 167us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34284: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":32}
W20260929 10:50:20.629344 26337 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34284: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260929 10:50:20.629477 26364 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:20.173925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37883 (local address 127.0.0.1:34284)
0929 10:50:20.174085 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:20.174095 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:20.174154 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:20.174321 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:20.174322 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:20.174324 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0929 10:50:20.174383 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:20.174385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:20.175296 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:20.175297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:20.175893 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:20.175894 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:20.176354 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:20.176354 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:20.176402 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:20.176407 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 10:50:20.629197 (+452790us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0929 10:50:20.629329 (+ 132us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.80.254:37883: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":91,"threads_started":1}
I20260929 10:50:20.629819 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 26267
2026-09-29T10:50:20Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (654 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (89 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (90 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:50:20 dist-test-slave-vqch krb5kdc[26376](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:50:20 dist-test-slave-vqch krb5kdc[26376](info): set up 2 sockets
Sep 29 10:50:20 dist-test-slave-vqch krb5kdc[26376](info): commencing operation
krb5kdc: starting...
W20260929 10:50:21.840950 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:50:21 dist-test-slave-vqch krb5kdc[26376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679021, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:50:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:50:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.80.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
I20260929 10:50:21.965080 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:37395
--webserver_interface=127.20.80.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:42993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:37395
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:22.046854 26389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:22.047058 26389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:22.047081 26389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:22.048185 26389 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:50:22.048218 26389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:22.048233 26389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:22.048244 26389 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:50:22.048256 26389 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:50:22.049634 26389 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:42993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:37395
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:37395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:22.049819 26389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:22.049988 26389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:22.054711 26394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:22.054704 26395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:22.054805 26389 server_base.cc:1034] running on GCE node
W20260929 10:50:22.054703 26397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:22.055169 26389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:22.055481 26389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:22.056677 26389 hybrid_clock.cc:648] HybridClock initialized: now 1790679022056671 us; error 35 us; skew 500 ppm
Sep 29 10:50:22 dist-test-slave-vqch krb5kdc[26376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679022, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:22.060055 26389 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:50:22.060581 26389 webserver.cc:460] Webserver started at http://127.20.80.254:37833/ using document root <none> and password file <none>
I20260929 10:50:22.060817 26389 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:22.060861 26389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:22.061018 26389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:22.062079 26389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "3c2782cc0ccf4c9c944ad17e391520bb"
format_stamp: "Formatted at 2026-09-29 10:50:22 on dist-test-slave-vqch"
server_key: "180ca273eb2aeaec5d73423f3b6b27ea"
server_key_iv: "e5aabb777b1f88725f95a21cf7eb7dfc"
server_key_version: "encryptionkey@0"
I20260929 10:50:22.062450 26389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "3c2782cc0ccf4c9c944ad17e391520bb"
format_stamp: "Formatted at 2026-09-29 10:50:22 on dist-test-slave-vqch"
server_key: "180ca273eb2aeaec5d73423f3b6b27ea"
server_key_iv: "e5aabb777b1f88725f95a21cf7eb7dfc"
server_key_version: "encryptionkey@0"
I20260929 10:50:22.063796 26389 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 10:50:22.064496 26404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:22.064654 26389 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:22.064759 26389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "3c2782cc0ccf4c9c944ad17e391520bb"
format_stamp: "Formatted at 2026-09-29 10:50:22 on dist-test-slave-vqch"
server_key: "180ca273eb2aeaec5d73423f3b6b27ea"
server_key_iv: "e5aabb777b1f88725f95a21cf7eb7dfc"
server_key_version: "encryptionkey@0"
I20260929 10:50:22.064831 26389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:22.080209 26389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:22.082125 26389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:22.082290 26389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:22.086357 26389 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:37395
I20260929 10:50:22.086393 26456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:37395 every 8 connection(s)
I20260929 10:50:22.086746 26389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:50:22.087641 26457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:22.089993 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 26389
I20260929 10:50:22.090035 26457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c2782cc0ccf4c9c944ad17e391520bb: Bootstrap starting.
I20260929 10:50:22.090098 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:50:22.090276 20803 external_mini_cluster.cc:1428] Setting key 32268859c100c0c67759681511410dc0
I20260929 10:50:22.090755 26457 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c2782cc0ccf4c9c944ad17e391520bb: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:22.091095 26457 log.cc:824] T 00000000000000000000000000000000 P 3c2782cc0ccf4c9c944ad17e391520bb: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:22.091841 26457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c2782cc0ccf4c9c944ad17e391520bb: No bootstrap required, opened a new log
I20260929 10:50:22.094208 26457 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c2782cc0ccf4c9c944ad17e391520bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c2782cc0ccf4c9c944ad17e391520bb" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37395 } }
I20260929 10:50:22.094378 26457 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c2782cc0ccf4c9c944ad17e391520bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:22.094408 26457 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c2782cc0ccf4c9c944ad17e391520bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c2782cc0ccf4c9c944ad17e391520bb, State: Initialized, Role: FOLLOWER
I20260929 10:50:22.094522 26457 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c2782cc0ccf4c9c944ad17e391520bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c2782cc0ccf4c9c944ad17e391520bb" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37395 } }
I20260929 10:50:22.094614 26457 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c2782cc0ccf4c9c944ad17e391520bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:50:22.094647 26457 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c2782cc0ccf4c9c944ad17e391520bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:50:22.094695 26457 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c2782cc0ccf4c9c944ad17e391520bb [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:22.095266 26457 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c2782cc0ccf4c9c944ad17e391520bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c2782cc0ccf4c9c944ad17e391520bb" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37395 } }
I20260929 10:50:22.095388 26457 leader_election.cc:302] T 00000000000000000000000000000000 P 3c2782cc0ccf4c9c944ad17e391520bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c2782cc0ccf4c9c944ad17e391520bb; no voters:
I20260929 10:50:22.095544 26457 leader_election.cc:288] T 00000000000000000000000000000000 P 3c2782cc0ccf4c9c944ad17e391520bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:50:22.095711 26463 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c2782cc0ccf4c9c944ad17e391520bb [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:22.095801 26457 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c2782cc0ccf4c9c944ad17e391520bb [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:50:22.095908 26463 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c2782cc0ccf4c9c944ad17e391520bb [term 1 LEADER]: Becoming Leader. State: Replica: 3c2782cc0ccf4c9c944ad17e391520bb, State: Running, Role: LEADER
I20260929 10:50:22.096031 26463 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c2782cc0ccf4c9c944ad17e391520bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c2782cc0ccf4c9c944ad17e391520bb" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37395 } }
I20260929 10:50:22.096391 26464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c2782cc0ccf4c9c944ad17e391520bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c2782cc0ccf4c9c944ad17e391520bb. Latest consensus state: current_term: 1 leader_uuid: "3c2782cc0ccf4c9c944ad17e391520bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c2782cc0ccf4c9c944ad17e391520bb" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37395 } } }
I20260929 10:50:22.096486 26464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c2782cc0ccf4c9c944ad17e391520bb [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:22.096403 26462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c2782cc0ccf4c9c944ad17e391520bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c2782cc0ccf4c9c944ad17e391520bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c2782cc0ccf4c9c944ad17e391520bb" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37395 } } }
I20260929 10:50:22.096571 26462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c2782cc0ccf4c9c944ad17e391520bb [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:22.097230 26469 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 29 10:50:22 dist-test-slave-vqch krb5kdc[26376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679021, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:22.097703 26469 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:50:22.099323 26469 catalog_manager.cc:1347] Generated new cluster ID: 6d4b4aff818e425bb9a688ec4d91b5ea
I20260929 10:50:22.099375 26469 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:50:22.101809 26460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:22.090794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48588 (local address 127.20.80.254:37395)
0929 10:50:22.090984 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:22.090992 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:22.091017 (+ 25us) server_negotiation.cc:407] Connection header received
0929 10:50:22.091616 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:22.091624 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:22.091673 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:22.091788 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:22.092598 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:22.093527 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:22.094600 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:22.095096 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:22.095174 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:22.098159 (+ 2985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:22.098163 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:22.098164 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:22.098174 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:22.100173 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:22.100563 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:22.100566 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:22.100566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:22.100674 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:22.100879 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:22.100880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:22.100880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:22.101216 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:22.101392 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:22.101468 (+ 76us) server_negotiation.cc:299] Negotiation successful
0929 10:50:22.101600 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":93,"threads_started":1}
I20260929 10:50:22.108352 26469 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:50:22.108916 26469 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:50:22.113422 26469 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c2782cc0ccf4c9c944ad17e391520bb: Generated new TSK 0
I20260929 10:50:22.113583 26469 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.80.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
I20260929 10:50:22.144366 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.193:0
--local_ip_for_outbound_sockets=127.20.80.193
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37395
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:42993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:22.225283 26483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:22.225526 26483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:22.225564 26483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:22.226732 26483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:22.226796 26483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:22.226842 26483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.193
I20260929 10:50:22.228345 26483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:42993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.193
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:22.228609 26483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:22.228847 26483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:22.233798 26488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:22.233834 26489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:22.234041 26483 server_base.cc:1034] running on GCE node
W20260929 10:50:22.233809 26491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:22.234332 26483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:22.234548 26483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:22.235688 26483 hybrid_clock.cc:648] HybridClock initialized: now 1790679022235681 us; error 33 us; skew 500 ppm
Sep 29 10:50:22 dist-test-slave-vqch krb5kdc[26376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679022, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:22.238184 26483 init.cc:377] Logged in from keytab as kudu/127.20.80.193@KRBTEST.COM (short username kudu)
I20260929 10:50:22.238615 26483 webserver.cc:460] Webserver started at http://127.20.80.193:43989/ using document root <none> and password file <none>
I20260929 10:50:22.238840 26483 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:22.238929 26483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:22.239073 26483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:22.240113 26483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-0/data/instance:
uuid: "42f7a1108d16427690b97198ebf72c33"
format_stamp: "Formatted at 2026-09-29 10:50:22 on dist-test-slave-vqch"
server_key: "5dd6af9155a1d517c794ff0a263ab501"
server_key_iv: "514702f9278a927631f28d396b35d139"
server_key_version: "encryptionkey@0"
I20260929 10:50:22.240479 26483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance:
uuid: "42f7a1108d16427690b97198ebf72c33"
format_stamp: "Formatted at 2026-09-29 10:50:22 on dist-test-slave-vqch"
server_key: "5dd6af9155a1d517c794ff0a263ab501"
server_key_iv: "514702f9278a927631f28d396b35d139"
server_key_version: "encryptionkey@0"
I20260929 10:50:22.241891 26483 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 10:50:22.242622 26498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:22.242810 26483 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:50:22.242919 26483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-0/wal
uuid: "42f7a1108d16427690b97198ebf72c33"
format_stamp: "Formatted at 2026-09-29 10:50:22 on dist-test-slave-vqch"
server_key: "5dd6af9155a1d517c794ff0a263ab501"
server_key_iv: "514702f9278a927631f28d396b35d139"
server_key_version: "encryptionkey@0"
I20260929 10:50:22.242993 26483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:22.257555 26483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:22.259333 26483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:22.259495 26483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:22.259714 26483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:22.260011 26483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:22.260064 26483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:22.260105 26483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:22.260135 26483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:22.265885 26483 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.193:40171
I20260929 10:50:22.265936 26612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.193:40171 every 8 connection(s)
I20260929 10:50:22.266222 26483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
I20260929 10:50:22.269455 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 26483
I20260929 10:50:22.269557 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance
I20260929 10:50:22.269733 20803 external_mini_cluster.cc:1428] Setting key 77fc85bb7f8bff3dedbed5200c109f2b
Sep 29 10:50:22 dist-test-slave-vqch krb5kdc[26376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679022, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:22.276132 26460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:22.266915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:42347 (local address 127.20.80.254:37395)
0929 10:50:22.266980 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:22.266982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:22.267182 (+ 200us) server_negotiation.cc:407] Connection header received
0929 10:50:22.268081 (+ 899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:22.268082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:22.268092 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:22.268142 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:22.268691 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:22.269221 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:22.270620 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:22.270818 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:22.270869 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:22.273382 (+ 2513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:22.273384 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:22.273385 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:22.273393 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:22.274780 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:22.275191 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:22.275192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:22.275193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:22.275249 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:22.275505 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:22.275506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:22.275506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:22.275615 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:22.275684 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:22.275963 (+ 279us) server_negotiation.cc:299] Negotiation successful
0929 10:50:22.276043 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 10:50:22.276311 26615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:22.266919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37395 (local address 127.20.80.193:42347)
0929 10:50:22.267067 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:22.267101 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:22.268007 (+ 906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:22.268192 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:22.268196 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:22.268441 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:22.268618 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:22.268622 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:22.269307 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:22.269308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:22.270533 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:22.270536 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:22.270900 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:22.270902 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:22.271052 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:22.271552 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:22.271562 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:22.273289 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:22.274871 (+ 1582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:22.274879 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:22.274881 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:22.275101 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:22.275322 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:22.275322 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:22.275323 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:22.275428 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:22.275689 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:22.275696 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:22.275826 (+ 130us) client_negotiation.cc:769] Sending connection context
0929 10:50:22.275930 (+ 104us) client_negotiation.cc:240] Negotiation successful
0929 10:50:22.276076 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":82,"threads_started":1}
I20260929 10:50:22.276780 26613 heartbeater.cc:347] Connected to a master server at 127.20.80.254:37395
I20260929 10:50:22.276890 26613 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:22.277107 26613 heartbeater.cc:511] Master 127.20.80.254:37395 requested a full tablet report, sending...
I20260929 10:50:22.277720 26421 ts_manager.cc:194] Registered new tserver with Master: 42f7a1108d16427690b97198ebf72c33 (127.20.80.193:40171)
WARNING: no policy specified for kudu/127.20.80.194@KRBTEST.COM; defaulting to no policy
I20260929 10:50:22.278605 26421 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.193@KRBTEST.COM'} at 127.20.80.193:42347
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
I20260929 10:50:22.295490 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.194:0
--local_ip_for_outbound_sockets=127.20.80.194
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37395
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:42993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:22.375289 26619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:22.375492 26619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:22.375515 26619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:22.376664 26619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:22.376703 26619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:22.376729 26619 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.194
I20260929 10:50:22.378221 26619 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:42993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.194
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:22.378473 26619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:22.378693 26619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 10:50:22.383716 26619 server_base.cc:1034] running on GCE node
W20260929 10:50:22.383663 26627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:22.383682 26625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:22.383659 26624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:22.384083 26619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:22.384344 26619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:22.385486 26619 hybrid_clock.cc:648] HybridClock initialized: now 1790679022385472 us; error 30 us; skew 500 ppm
Sep 29 10:50:22 dist-test-slave-vqch krb5kdc[26376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679022, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:22.387905 26619 init.cc:377] Logged in from keytab as kudu/127.20.80.194@KRBTEST.COM (short username kudu)
I20260929 10:50:22.388355 26619 webserver.cc:460] Webserver started at http://127.20.80.194:35897/ using document root <none> and password file <none>
I20260929 10:50:22.388593 26619 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:22.388636 26619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:22.388803 26619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:22.389932 26619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-1/data/instance:
uuid: "996474f25c4f401a89bde1124160cda8"
format_stamp: "Formatted at 2026-09-29 10:50:22 on dist-test-slave-vqch"
server_key: "aa6528dbdd95b4d36042026125bc1608"
server_key_iv: "c6a10ab030164022d24a5c0ba911d86d"
server_key_version: "encryptionkey@0"
I20260929 10:50:22.390300 26619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance:
uuid: "996474f25c4f401a89bde1124160cda8"
format_stamp: "Formatted at 2026-09-29 10:50:22 on dist-test-slave-vqch"
server_key: "aa6528dbdd95b4d36042026125bc1608"
server_key_iv: "c6a10ab030164022d24a5c0ba911d86d"
server_key_version: "encryptionkey@0"
I20260929 10:50:22.391602 26619 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:50:22.392352 26634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:22.392534 26619 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:22.392630 26619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-1/wal
uuid: "996474f25c4f401a89bde1124160cda8"
format_stamp: "Formatted at 2026-09-29 10:50:22 on dist-test-slave-vqch"
server_key: "aa6528dbdd95b4d36042026125bc1608"
server_key_iv: "c6a10ab030164022d24a5c0ba911d86d"
server_key_version: "encryptionkey@0"
I20260929 10:50:22.392715 26619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:22.427563 26619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:22.429636 26619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:22.429807 26619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:22.430054 26619 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:22.430431 26619 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:22.430487 26619 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:22.430531 26619 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:22.430559 26619 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:22.436252 26619 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.194:38853
I20260929 10:50:22.436299 26748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.194:38853 every 8 connection(s)
I20260929 10:50:22.436648 26619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
I20260929 10:50:22.440667 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 26619
I20260929 10:50:22.440757 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance
I20260929 10:50:22.441012 20803 external_mini_cluster.cc:1428] Setting key 804f02f1f7bf9ef94a68284b0f963c22
Sep 29 10:50:22 dist-test-slave-vqch krb5kdc[26376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679022, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:22.446619 26460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:22.437441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:46811 (local address 127.20.80.254:37395)
0929 10:50:22.437502 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:22.437504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:22.437719 (+ 215us) server_negotiation.cc:407] Connection header received
0929 10:50:22.438650 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:22.438650 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:22.438659 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:22.438718 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:22.439329 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:22.439853 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:22.440967 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:22.441282 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:22.441359 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:22.443763 (+ 2404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:22.443764 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:22.443764 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:22.443772 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:22.445191 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:22.445649 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:22.445650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:22.445650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:22.445701 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:22.445973 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:22.445974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:22.445974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:22.446089 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:22.446160 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:22.446448 (+ 288us) server_negotiation.cc:299] Negotiation successful
0929 10:50:22.446530 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260929 10:50:22.446808 26751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:22.437430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37395 (local address 127.20.80.194:46811)
0929 10:50:22.437587 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:22.437629 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:22.438562 (+ 933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:22.438774 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:22.438778 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:22.439032 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:22.439222 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:22.439226 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:22.439971 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:22.439972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:22.440874 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:22.440878 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:22.441398 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:22.441399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:22.441549 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:22.442161 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:22.442170 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:22.443665 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:22.445289 (+ 1624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:22.445295 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:22.445298 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:22.445521 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:22.445783 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:22.445783 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:22.445784 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:22.445903 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:22.446162 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:22.446170 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:22.446311 (+ 141us) client_negotiation.cc:769] Sending connection context
0929 10:50:22.446412 (+ 101us) client_negotiation.cc:240] Negotiation successful
0929 10:50:22.446569 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":89,"threads_started":1}
I20260929 10:50:22.447288 26749 heartbeater.cc:347] Connected to a master server at 127.20.80.254:37395
I20260929 10:50:22.447379 26749 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:22.447556 26749 heartbeater.cc:511] Master 127.20.80.254:37395 requested a full tablet report, sending...
I20260929 10:50:22.447911 26421 ts_manager.cc:194] Registered new tserver with Master: 996474f25c4f401a89bde1124160cda8 (127.20.80.194:38853)
I20260929 10:50:22.448300 26421 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.194@KRBTEST.COM'} at 127.20.80.194:46811
WARNING: no policy specified for kudu/127.20.80.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
I20260929 10:50:22.466101 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.195:0
--local_ip_for_outbound_sockets=127.20.80.195
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37395
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:42993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:22.547078 26754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:22.547278 26754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:22.547302 26754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:22.548372 26754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:22.548405 26754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:22.548430 26754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.195
I20260929 10:50:22.549809 26754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:42993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.195
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:22.549985 26754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:22.550208 26754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:22.555121 26760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:22.555125 26759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:22.555176 26762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:22.555346 26754 server_base.cc:1034] running on GCE node
I20260929 10:50:22.555576 26754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:22.555801 26754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:22.556943 26754 hybrid_clock.cc:648] HybridClock initialized: now 1790679022556929 us; error 26 us; skew 500 ppm
Sep 29 10:50:22 dist-test-slave-vqch krb5kdc[26376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679022, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:22.559508 26754 init.cc:377] Logged in from keytab as kudu/127.20.80.195@KRBTEST.COM (short username kudu)
I20260929 10:50:22.559937 26754 webserver.cc:460] Webserver started at http://127.20.80.195:37067/ using document root <none> and password file <none>
I20260929 10:50:22.560137 26754 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:22.560177 26754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:22.560338 26754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:22.561376 26754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-2/data/instance:
uuid: "e5bdcbcf2a754bed9bfcb2fb2d87473d"
format_stamp: "Formatted at 2026-09-29 10:50:22 on dist-test-slave-vqch"
server_key: "3f2b1dbd9f5bf975264fbfdc19aaffb5"
server_key_iv: "09281b1e40a1f77209c7cc1093d2c22c"
server_key_version: "encryptionkey@0"
I20260929 10:50:22.561668 26754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance:
uuid: "e5bdcbcf2a754bed9bfcb2fb2d87473d"
format_stamp: "Formatted at 2026-09-29 10:50:22 on dist-test-slave-vqch"
server_key: "3f2b1dbd9f5bf975264fbfdc19aaffb5"
server_key_iv: "09281b1e40a1f77209c7cc1093d2c22c"
server_key_version: "encryptionkey@0"
I20260929 10:50:22.562857 26754 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:22.563503 26770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:22.563669 26754 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:22.563756 26754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-2/wal
uuid: "e5bdcbcf2a754bed9bfcb2fb2d87473d"
format_stamp: "Formatted at 2026-09-29 10:50:22 on dist-test-slave-vqch"
server_key: "3f2b1dbd9f5bf975264fbfdc19aaffb5"
server_key_iv: "09281b1e40a1f77209c7cc1093d2c22c"
server_key_version: "encryptionkey@0"
I20260929 10:50:22.563835 26754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:22.576411 26754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:22.578701 26754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:22.578857 26754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:22.579088 26754 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:22.579404 26754 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:22.579458 26754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:22.579499 26754 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:22.579527 26754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:22.585565 26754 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.195:38521
I20260929 10:50:22.585624 26883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.195:38521 every 8 connection(s)
I20260929 10:50:22.585912 26754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
I20260929 10:50:22.590761 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 26754
I20260929 10:50:22.590862 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance
I20260929 10:50:22.591059 20803 external_mini_cluster.cc:1428] Setting key 15013797b571d35f0c6595f63380d59f
Sep 29 10:50:22 dist-test-slave-vqch krb5kdc[26376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679022, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:22.595036 26460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:22.586568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:54867 (local address 127.20.80.254:37395)
0929 10:50:22.586609 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:22.586611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:22.586796 (+ 185us) server_negotiation.cc:407] Connection header received
0929 10:50:22.587691 (+ 895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:22.587692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:22.587701 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:22.587763 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:22.588355 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:22.588878 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:22.589900 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:22.590093 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:22.590149 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:22.592537 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:22.592538 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:22.592540 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:22.592551 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:22.593946 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:22.594290 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:22.594291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:22.594291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:22.594338 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:22.594544 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:22.594545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:22.594545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:22.594644 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:22.594691 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:22.594916 (+ 225us) server_negotiation.cc:299] Negotiation successful
0929 10:50:22.594973 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 10:50:22.595340 26886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:22.586516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37395 (local address 127.20.80.195:54867)
0929 10:50:22.586670 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:22.586711 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:22.587604 (+ 893us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:22.587837 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:22.587841 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:22.588086 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:22.588264 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:22.588268 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:22.588955 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:22.588955 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:22.589813 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:22.589816 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:22.590168 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:22.590168 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:22.590266 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:22.590799 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:22.590807 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:22.592445 (+ 1638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:22.594011 (+ 1566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:22.594017 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:22.594019 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:22.594226 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:22.594397 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:22.594397 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:22.594398 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:22.594491 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:22.594696 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:22.594699 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:22.594810 (+ 111us) client_negotiation.cc:769] Sending connection context
0929 10:50:22.594895 (+ 85us) client_negotiation.cc:240] Negotiation successful
0929 10:50:22.595086 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260929 10:50:22.595788 26884 heartbeater.cc:347] Connected to a master server at 127.20.80.254:37395
I20260929 10:50:22.595906 26884 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:22.596133 26884 heartbeater.cc:511] Master 127.20.80.254:37395 requested a full tablet report, sending...
I20260929 10:50:22.596580 26421 ts_manager.cc:194] Registered new tserver with Master: e5bdcbcf2a754bed9bfcb2fb2d87473d (127.20.80.195:38521)
I20260929 10:50:22.597060 26421 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.195@KRBTEST.COM'} at 127.20.80.195:54867
I20260929 10:50:22.603513 20803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/test-admin.keytab.
Sep 29 10:50:22 dist-test-slave-vqch krb5kdc[26376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:22.617520 20803 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260929 10:50:22.619185 26891 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:22.618446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47102 (local address 127.20.80.193:40171)
0929 10:50:22.618609 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:22.618613 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:22.618623 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:50:22.618649 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:22.618651 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:22.618663 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:22.618732 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:22.619094 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47102: BlockingRecv error: recv got EOF from 127.0.0.1:47102 (error 108)
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":85,"threads_started":1}
W20260929 10:50:22.619246 20803 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/krb5cc' of type FILE
Sep 29 10:50:22 dist-test-slave-vqch krb5kdc[26376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:22.620151 20803 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 10:50:22 dist-test-slave-vqch krb5kdc[26376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:22.620966 20803 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 10:50:22 dist-test-slave-vqch krb5kdc[26376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.193@KRBTEST.COM
I20260929 10:50:22.628401 26891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:22.621513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47114 (local address 127.20.80.193:40171)
0929 10:50:22.621590 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:22.621591 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:22.621635 (+ 44us) server_negotiation.cc:407] Connection header received
0929 10:50:22.621674 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:22.621674 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:22.621683 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:22.621729 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:22.622084 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:22.622773 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:22.623482 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:22.623781 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:22.623832 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:22.625803 (+ 1971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:22.625811 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:22.625811 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:22.625817 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:22.627354 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:22.627687 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:22.627690 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:22.627690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:22.627742 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:22.627905 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:22.627905 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:22.627906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:22.628023 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:22.628118 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:22.628183 (+ 65us) server_negotiation.cc:299] Negotiation successful
0929 10:50:22.628241 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260929 10:50:22.630224 26891 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:22.629637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47126 (local address 127.20.80.193:40171)
0929 10:50:22.629691 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:22.629693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:22.629718 (+ 25us) server_negotiation.cc:407] Connection header received
0929 10:50:22.629779 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:22.629779 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:22.629787 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:22.629834 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:22.630178 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47126: BlockingRecv error: recv got EOF from 127.0.0.1:47126 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260929 10:50:22.630276 20803 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/krb5cc' of type FILE
Sep 29 10:50:22 dist-test-slave-vqch krb5kdc[26376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:22.631125 20803 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 10:50:22 dist-test-slave-vqch krb5kdc[26376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:22.631897 20803 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 10:50:22 dist-test-slave-vqch krb5kdc[26376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.193@KRBTEST.COM
I20260929 10:50:22.638448 26891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:22.632417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47136 (local address 127.20.80.193:40171)
0929 10:50:22.632452 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:22.632453 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:22.632471 (+ 18us) server_negotiation.cc:407] Connection header received
0929 10:50:22.632518 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:22.632518 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:22.632525 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:22.632568 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:22.632898 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:22.633433 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:22.634042 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:22.634219 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:22.634264 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:22.636024 (+ 1760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:22.636025 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:22.636026 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:22.636031 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:22.637457 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:22.637811 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:22.637812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:22.637813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:22.637868 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:22.638029 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:22.638030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:22.638030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:22.638113 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:22.638181 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:22.638339 (+ 158us) server_negotiation.cc:299] Negotiation successful
0929 10:50:22.638394 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260929 10:50:22.639894 26891 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:22.639415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47152 (local address 127.20.80.193:40171)
0929 10:50:22.639450 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:22.639452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:22.639534 (+ 82us) server_negotiation.cc:407] Connection header received
0929 10:50:22.639565 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:22.639565 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:22.639572 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:22.639619 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:22.639827 (+ 208us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47152: BlockingRecv error: recv got EOF from 127.0.0.1:47152 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260929 10:50:22.640009 20803 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790678996126930-20803-0/krb5kdc/krb5cc' of type FILE
Sep 29 10:50:22 dist-test-slave-vqch krb5kdc[26376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:22.640901 20803 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 10:50:22 dist-test-slave-vqch krb5kdc[26376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:22.641856 20803 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 10:50:22 dist-test-slave-vqch krb5kdc[26376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.193@KRBTEST.COM
I20260929 10:50:22.648305 26891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:22.642336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47156 (local address 127.20.80.193:40171)
0929 10:50:22.642386 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:22.642388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:22.642410 (+ 22us) server_negotiation.cc:407] Connection header received
0929 10:50:22.642458 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:22.642459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:22.642465 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:22.642508 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:22.642847 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:22.643285 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:22.643943 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:22.644108 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:22.644156 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:22.646127 (+ 1971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:22.646129 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:22.646129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:22.646136 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:22.647446 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:22.647724 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:22.647725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:22.647725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:22.647769 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:22.647918 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:22.647919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:22.647919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:22.647988 (+ 69us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:22.648050 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:22.648130 (+ 80us) server_negotiation.cc:299] Negotiation successful
0929 10:50:22.648176 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260929 10:50:22.649215 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 26483
I20260929 10:50:22.654764 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 26619
I20260929 10:50:22.658983 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 26754
I20260929 10:50:22.663228 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 26389
2026-09-29T10:50:22Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1854 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:50:22 dist-test-slave-vqch krb5kdc[26904](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:50:22 dist-test-slave-vqch krb5kdc[26904](info): set up 2 sockets
Sep 29 10:50:22 dist-test-slave-vqch krb5kdc[26904](info): commencing operation
krb5kdc: starting...
W20260929 10:50:23.761402 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:50:23 dist-test-slave-vqch krb5kdc[26904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:50:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:50:23Z Disabled control of system clock
WARNING: no policy specified for oryx/127.20.80.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.80.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.80.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.254.keytab.
Entry for principal oryx/127.20.80.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.254.keytab.
I20260929 10:50:23.875957 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:44567
--webserver_interface=127.20.80.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.254.keytab
--principal=oryx/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:40769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:44567
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:23.966153 26918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:23.966388 26918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:23.966428 26918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:23.967578 26918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:50:23.967658 26918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:23.967690 26918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:23.967725 26918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:50:23.967751 26918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:50:23.969282 26918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:40769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:44567
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.254.keytab
--principal=oryx/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:44567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:23.969568 26918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:23.969806 26918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:23.974740 26924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:23.974721 26926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:23.974721 26923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:23.974978 26918 server_base.cc:1034] running on GCE node
I20260929 10:50:23.975201 26918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:23.975448 26918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:23.976585 26918 hybrid_clock.cc:648] HybridClock initialized: now 1790679023976573 us; error 26 us; skew 500 ppm
Sep 29 10:50:23 dist-test-slave-vqch krb5kdc[26904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679023, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:23.982481 26918 init.cc:377] Logged in from keytab as oryx/127.20.80.254@KRBTEST.COM (short username oryx)
I20260929 10:50:23.982862 26918 webserver.cc:460] Webserver started at http://127.20.80.254:39949/ using document root <none> and password file <none>
I20260929 10:50:23.983059 26918 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:23.983098 26918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:23.983186 26918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:23.984182 26918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "39326fe5440f4d74abd39e9b375c9eaf"
format_stamp: "Formatted at 2026-09-29 10:50:23 on dist-test-slave-vqch"
server_key: "dbe18c0360a55b68fb597d2a38adfaff"
server_key_iv: "e779aca31d88dd20f413d507d248c10e"
server_key_version: "encryptionkey@0"
I20260929 10:50:23.984488 26918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "39326fe5440f4d74abd39e9b375c9eaf"
format_stamp: "Formatted at 2026-09-29 10:50:23 on dist-test-slave-vqch"
server_key: "dbe18c0360a55b68fb597d2a38adfaff"
server_key_iv: "e779aca31d88dd20f413d507d248c10e"
server_key_version: "encryptionkey@0"
I20260929 10:50:23.985821 26918 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 10:50:23.986519 26933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:23.986759 26918 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:50:23.986841 26918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "39326fe5440f4d74abd39e9b375c9eaf"
format_stamp: "Formatted at 2026-09-29 10:50:23 on dist-test-slave-vqch"
server_key: "dbe18c0360a55b68fb597d2a38adfaff"
server_key_iv: "e779aca31d88dd20f413d507d248c10e"
server_key_version: "encryptionkey@0"
I20260929 10:50:23.986903 26918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:24.006109 26918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:24.007977 26918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:24.008095 26918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:24.012091 26918 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:44567
I20260929 10:50:24.012161 26985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:44567 every 8 connection(s)
I20260929 10:50:24.012529 26918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:50:24.013551 26986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:24.015733 26986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf: Bootstrap starting.
I20260929 10:50:24.016309 26986 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:24.016593 26986 log.cc:824] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:24.017405 26986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf: No bootstrap required, opened a new log
I20260929 10:50:24.019047 26986 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } }
I20260929 10:50:24.019187 26986 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:24.019233 26986 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39326fe5440f4d74abd39e9b375c9eaf, State: Initialized, Role: FOLLOWER
I20260929 10:50:24.019392 26986 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } }
I20260929 10:50:24.019452 26986 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:50:24.019474 26986 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:50:24.019543 26986 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:24.020335 26986 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } }
I20260929 10:50:24.020452 26986 leader_election.cc:302] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39326fe5440f4d74abd39e9b375c9eaf; no voters:
I20260929 10:50:24.020660 26986 leader_election.cc:288] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:50:24.020715 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 26918
I20260929 10:50:24.020686 26989 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:24.020843 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:50:24.020960 26989 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [term 1 LEADER]: Becoming Leader. State: Replica: 39326fe5440f4d74abd39e9b375c9eaf, State: Running, Role: LEADER
I20260929 10:50:24.021031 26986 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:50:24.021068 20803 external_mini_cluster.cc:1428] Setting key f1cba6294a8f7142d17357001287d0d5
I20260929 10:50:24.021147 26989 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } }
I20260929 10:50:24.021574 26991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39326fe5440f4d74abd39e9b375c9eaf. Latest consensus state: current_term: 1 leader_uuid: "39326fe5440f4d74abd39e9b375c9eaf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } }
I20260929 10:50:24.021670 26991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:24.021833 26990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39326fe5440f4d74abd39e9b375c9eaf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } }
I20260929 10:50:24.021901 26990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:24.021958 27000 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:50:24.022455 27000 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:50:24.024036 27000 catalog_manager.cc:1347] Generated new cluster ID: 3fef4c4dad1e479d988b0cb330c76f15
I20260929 10:50:24.024087 27000 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 10:50:24 dist-test-slave-vqch krb5kdc[26904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM
I20260929 10:50:24.031276 27000 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:50:24.031855 26998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:24.021658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52434 (local address 127.20.80.254:44567)
0929 10:50:24.021861 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:24.021870 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:24.021906 (+ 36us) server_negotiation.cc:407] Connection header received
0929 10:50:24.022023 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:24.022030 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:24.022081 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:24.022222 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:24.023074 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.024353 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.025171 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.025592 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.025710 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:24.027937 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:24.027941 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:24.027941 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:24.027952 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:24.030148 (+ 2196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:24.030533 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:24.030536 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:24.030536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:24.030634 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:24.030997 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:24.030998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:24.030999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:24.031287 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:24.031473 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:24.031538 (+ 65us) server_negotiation.cc:299] Negotiation successful
0929 10:50:24.031676 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":77,"threads_started":1}
I20260929 10:50:24.031989 27000 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:50:24.034554 27000 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf: Generated new TSK 0
I20260929 10:50:24.034716 27000 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.20.80.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.80.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.80.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.193.keytab.
Entry for principal oryx/127.20.80.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.193.keytab.
I20260929 10:50:24.066009 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.193:0
--local_ip_for_outbound_sockets=127.20.80.193
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:44567
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.193.keytab
--principal=oryx/127.20.80.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:40769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:24.170694 27012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:24.170935 27012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:24.171135 27012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:24.172312 27012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:24.172380 27012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:24.172442 27012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.193
I20260929 10:50:24.173969 27012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:40769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.193.keytab
--principal=oryx/127.20.80.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:44567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.193
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:24.174224 27012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:24.174456 27012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:24.179486 27020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:24.179652 27012 server_base.cc:1034] running on GCE node
W20260929 10:50:24.179493 27017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:24.179586 27018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:24.180073 27012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:24.180331 27012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:24.181478 27012 hybrid_clock.cc:648] HybridClock initialized: now 1790679024181467 us; error 27 us; skew 500 ppm
Sep 29 10:50:24 dist-test-slave-vqch krb5kdc[26904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679024, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:24.184346 27012 init.cc:377] Logged in from keytab as oryx/127.20.80.193@KRBTEST.COM (short username oryx)
I20260929 10:50:24.184947 27012 webserver.cc:460] Webserver started at http://127.20.80.193:34809/ using document root <none> and password file <none>
I20260929 10:50:24.185297 27012 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:24.185365 27012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:24.185499 27012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:24.186961 27012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-0/data/instance:
uuid: "3da20f0055b74c0292fa05c61074e4d6"
format_stamp: "Formatted at 2026-09-29 10:50:24 on dist-test-slave-vqch"
server_key: "65a9e623a43fc0b0f4b7f4f6048c2c0f"
server_key_iv: "8bf6bedac128355cdf306bb929424d88"
server_key_version: "encryptionkey@0"
I20260929 10:50:24.187386 27012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance:
uuid: "3da20f0055b74c0292fa05c61074e4d6"
format_stamp: "Formatted at 2026-09-29 10:50:24 on dist-test-slave-vqch"
server_key: "65a9e623a43fc0b0f4b7f4f6048c2c0f"
server_key_iv: "8bf6bedac128355cdf306bb929424d88"
server_key_version: "encryptionkey@0"
I20260929 10:50:24.189332 27012 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20260929 10:50:24.190259 27027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:24.190469 27012 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:24.190563 27012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-0/wal
uuid: "3da20f0055b74c0292fa05c61074e4d6"
format_stamp: "Formatted at 2026-09-29 10:50:24 on dist-test-slave-vqch"
server_key: "65a9e623a43fc0b0f4b7f4f6048c2c0f"
server_key_iv: "8bf6bedac128355cdf306bb929424d88"
server_key_version: "encryptionkey@0"
I20260929 10:50:24.190635 27012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:24.207159 27012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:24.209620 27012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:24.209748 27012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:24.209935 27012 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:24.210186 27012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:24.210223 27012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:24.210247 27012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:24.210261 27012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:24.216357 27012 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.193:41319
I20260929 10:50:24.216434 27141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.193:41319 every 8 connection(s)
I20260929 10:50:24.216790 27012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
I20260929 10:50:24.221716 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 27012
I20260929 10:50:24.221841 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance
I20260929 10:50:24.222021 20803 external_mini_cluster.cc:1428] Setting key 4f83cc098e15ea9ade9ddedc2ea60625
Sep 29 10:50:24 dist-test-slave-vqch krb5kdc[26904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679024, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM
I20260929 10:50:24.226780 26998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:24.217533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:43549 (local address 127.20.80.254:44567)
0929 10:50:24.217594 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:24.217597 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:24.217800 (+ 203us) server_negotiation.cc:407] Connection header received
0929 10:50:24.218337 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:24.218338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:24.218360 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:24.218415 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:24.219347 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.219859 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.220901 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.221181 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.221257 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:24.223881 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:24.223882 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:24.223883 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:24.223891 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:24.225414 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:24.225873 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:24.225873 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:24.225874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:24.225929 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:24.226196 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:24.226196 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:24.226197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:24.226294 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:24.226347 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:24.226654 (+ 307us) server_negotiation.cc:299] Negotiation successful
0929 10:50:24.226718 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260929 10:50:24.227067 27144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:24.217502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.20.80.193:43549)
0929 10:50:24.217656 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:24.217691 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:24.218220 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:24.218501 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:24.218505 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:24.219053 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:24.219237 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:24.219242 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.219949 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.219950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:24.220799 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:24.220801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.221264 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.221265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:24.221396 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:24.221964 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:24.221980 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:24.223780 (+ 1800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:24.225494 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:24.225501 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:24.225503 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:24.225794 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:24.225998 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:24.225999 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:24.225999 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:24.226135 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:24.226358 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:24.226364 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:24.226521 (+ 157us) client_negotiation.cc:769] Sending connection context
0929 10:50:24.226636 (+ 115us) client_negotiation.cc:240] Negotiation successful
0929 10:50:24.226808 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":71,"threads_started":1}
I20260929 10:50:24.227612 27142 heartbeater.cc:347] Connected to a master server at 127.20.80.254:44567
I20260929 10:50:24.227708 27142 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:24.227913 27142 heartbeater.cc:511] Master 127.20.80.254:44567 requested a full tablet report, sending...
I20260929 10:50:24.228513 26950 ts_manager.cc:194] Registered new tserver with Master: 3da20f0055b74c0292fa05c61074e4d6 (127.20.80.193:41319)
I20260929 10:50:24.229496 26950 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.80.193@KRBTEST.COM'} at 127.20.80.193:43549
WARNING: no policy specified for oryx/127.20.80.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.80.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.80.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.194.keytab.
Entry for principal oryx/127.20.80.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.194.keytab.
I20260929 10:50:24.248055 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.194:0
--local_ip_for_outbound_sockets=127.20.80.194
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:44567
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.194.keytab
--principal=oryx/127.20.80.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:40769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:24.327369 27147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:24.327610 27147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:24.327646 27147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:24.328804 27147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:24.328874 27147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:24.328919 27147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.194
I20260929 10:50:24.330427 27147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:40769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.194.keytab
--principal=oryx/127.20.80.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:44567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.194
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:24.330696 27147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:24.330927 27147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:24.335928 27155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:24.335958 27153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:24.335928 27152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:24.336129 27147 server_base.cc:1034] running on GCE node
I20260929 10:50:24.336377 27147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:24.336581 27147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:24.337742 27147 hybrid_clock.cc:648] HybridClock initialized: now 1790679024337720 us; error 36 us; skew 500 ppm
Sep 29 10:50:24 dist-test-slave-vqch krb5kdc[26904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679024, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:24.340312 27147 init.cc:377] Logged in from keytab as oryx/127.20.80.194@KRBTEST.COM (short username oryx)
I20260929 10:50:24.340734 27147 webserver.cc:460] Webserver started at http://127.20.80.194:45883/ using document root <none> and password file <none>
I20260929 10:50:24.341028 27147 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:24.341107 27147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:24.341241 27147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:24.342319 27147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-1/data/instance:
uuid: "68952289b9a242f7a23f07cbcb8b36f2"
format_stamp: "Formatted at 2026-09-29 10:50:24 on dist-test-slave-vqch"
server_key: "4795e1c26477f56d11ba6e568da2d57d"
server_key_iv: "ebb66db63f488b19154f74457b1dda75"
server_key_version: "encryptionkey@0"
I20260929 10:50:24.342684 27147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance:
uuid: "68952289b9a242f7a23f07cbcb8b36f2"
format_stamp: "Formatted at 2026-09-29 10:50:24 on dist-test-slave-vqch"
server_key: "4795e1c26477f56d11ba6e568da2d57d"
server_key_iv: "ebb66db63f488b19154f74457b1dda75"
server_key_version: "encryptionkey@0"
I20260929 10:50:24.343973 27147 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:50:24.344668 27162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:24.344828 27147 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:24.344923 27147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-1/wal
uuid: "68952289b9a242f7a23f07cbcb8b36f2"
format_stamp: "Formatted at 2026-09-29 10:50:24 on dist-test-slave-vqch"
server_key: "4795e1c26477f56d11ba6e568da2d57d"
server_key_iv: "ebb66db63f488b19154f74457b1dda75"
server_key_version: "encryptionkey@0"
I20260929 10:50:24.345038 27147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:24.367460 27147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:24.369359 27147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:24.369518 27147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:24.369753 27147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:24.370065 27147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:24.370116 27147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:24.370152 27147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:24.370179 27147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:24.375901 27147 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.194:45001
I20260929 10:50:24.375995 27276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.194:45001 every 8 connection(s)
I20260929 10:50:24.376287 27147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
Sep 29 10:50:24 dist-test-slave-vqch krb5kdc[26904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679024, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM
I20260929 10:50:24.382612 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 27147
I20260929 10:50:24.382710 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance
I20260929 10:50:24.382908 20803 external_mini_cluster.cc:1428] Setting key 6dbfcbe84e5ddf473b90447ca788ff57
I20260929 10:50:24.385502 26998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:24.377030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:48043 (local address 127.20.80.254:44567)
0929 10:50:24.377091 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:24.377094 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:24.377285 (+ 191us) server_negotiation.cc:407] Connection header received
0929 10:50:24.377753 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:24.377754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:24.377764 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:24.377816 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:24.378691 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.379172 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.380132 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.380334 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.380389 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:24.382500 (+ 2111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:24.382501 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:24.382501 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:24.382508 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:24.383963 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:24.384462 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:24.384462 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:24.384463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:24.384511 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:24.384773 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:24.384774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:24.384774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:24.384870 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:24.384918 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:24.385288 (+ 370us) server_negotiation.cc:299] Negotiation successful
0929 10:50:24.385381 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 10:50:24.385530 27279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:24.376965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.20.80.194:48043)
0929 10:50:24.377170 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:24.377204 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:24.377675 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:24.377869 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:24.377872 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:24.378430 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:24.378618 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:24.378622 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.379251 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.379252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:24.380059 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:24.380061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.380407 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.380408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:24.380510 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:24.381066 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:24.381074 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:24.382374 (+ 1300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:24.384037 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:24.384043 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:24.384045 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:24.384381 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:24.384573 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:24.384574 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:24.384574 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:24.384714 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:24.384930 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:24.384936 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:24.385136 (+ 200us) client_negotiation.cc:769] Sending connection context
0929 10:50:24.385248 (+ 112us) client_negotiation.cc:240] Negotiation successful
0929 10:50:24.385372 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":80,"threads_started":1}
I20260929 10:50:24.386150 27277 heartbeater.cc:347] Connected to a master server at 127.20.80.254:44567
I20260929 10:50:24.386230 27277 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:24.386417 27277 heartbeater.cc:511] Master 127.20.80.254:44567 requested a full tablet report, sending...
I20260929 10:50:24.386765 26950 ts_manager.cc:194] Registered new tserver with Master: 68952289b9a242f7a23f07cbcb8b36f2 (127.20.80.194:45001)
I20260929 10:50:24.387148 26950 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.80.194@KRBTEST.COM'} at 127.20.80.194:48043
WARNING: no policy specified for oryx/127.20.80.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.80.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.80.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.195.keytab.
Entry for principal oryx/127.20.80.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.195.keytab.
I20260929 10:50:24.408030 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.195:0
--local_ip_for_outbound_sockets=127.20.80.195
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:44567
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.195.keytab
--principal=oryx/127.20.80.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:40769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:24.488292 27282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:24.488488 27282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:24.488511 27282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:24.489801 27282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:24.489841 27282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:24.489867 27282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.195
I20260929 10:50:24.491276 27282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:40769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.195.keytab
--principal=oryx/127.20.80.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:44567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.195
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:24.491504 27282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:24.491729 27282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:24.496837 27288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:24.496870 27290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:24.497053 27282 server_base.cc:1034] running on GCE node
W20260929 10:50:24.496851 27287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:24.497284 27282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:24.497521 27282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:24.498701 27282 hybrid_clock.cc:648] HybridClock initialized: now 1790679024498676 us; error 35 us; skew 500 ppm
Sep 29 10:50:24 dist-test-slave-vqch krb5kdc[26904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679024, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:24.501124 27282 init.cc:377] Logged in from keytab as oryx/127.20.80.195@KRBTEST.COM (short username oryx)
I20260929 10:50:24.501562 27282 webserver.cc:460] Webserver started at http://127.20.80.195:46135/ using document root <none> and password file <none>
I20260929 10:50:24.501798 27282 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:24.501866 27282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:24.501998 27282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:24.503062 27282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-2/data/instance:
uuid: "658a8336746b42189751b2817f40a623"
format_stamp: "Formatted at 2026-09-29 10:50:24 on dist-test-slave-vqch"
server_key: "56f89ef33ad2da08dcde4edea74964d9"
server_key_iv: "125e6484b2e51df6b4bda59af997d795"
server_key_version: "encryptionkey@0"
I20260929 10:50:24.503430 27282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance:
uuid: "658a8336746b42189751b2817f40a623"
format_stamp: "Formatted at 2026-09-29 10:50:24 on dist-test-slave-vqch"
server_key: "56f89ef33ad2da08dcde4edea74964d9"
server_key_iv: "125e6484b2e51df6b4bda59af997d795"
server_key_version: "encryptionkey@0"
I20260929 10:50:24.504710 27282 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 10:50:24.505599 27297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:24.505761 27282 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:24.505869 27282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-2/wal
uuid: "658a8336746b42189751b2817f40a623"
format_stamp: "Formatted at 2026-09-29 10:50:24 on dist-test-slave-vqch"
server_key: "56f89ef33ad2da08dcde4edea74964d9"
server_key_iv: "125e6484b2e51df6b4bda59af997d795"
server_key_version: "encryptionkey@0"
I20260929 10:50:24.505956 27282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:24.515012 27282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:24.516716 27282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:24.516870 27282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:24.517175 27282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:24.517494 27282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:24.517527 27282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:24.517549 27282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:24.517596 27282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:24.523726 27282 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.195:35313
I20260929 10:50:24.523797 27410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.195:35313 every 8 connection(s)
I20260929 10:50:24.524098 27282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
Sep 29 10:50:24 dist-test-slave-vqch krb5kdc[26904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679024, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM
I20260929 10:50:24.532604 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 27282
I20260929 10:50:24.532759 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance
I20260929 10:50:24.532915 26998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:24.524819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:45373 (local address 127.20.80.254:44567)
0929 10:50:24.524879 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:24.524881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:24.525083 (+ 202us) server_negotiation.cc:407] Connection header received
0929 10:50:24.525537 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:24.525538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:24.525547 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:24.525603 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:24.526473 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.526976 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.527885 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.528063 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.528112 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:24.530258 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:24.530260 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:24.530260 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:24.530268 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:24.531523 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:24.531952 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:24.531953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:24.531954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:24.532014 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:24.532266 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:24.532267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:24.532268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:24.532386 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:24.532494 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:24.532739 (+ 245us) server_negotiation.cc:299] Negotiation successful
0929 10:50:24.532826 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 10:50:24.532974 20803 external_mini_cluster.cc:1428] Setting key 7cd2b4d910f8f022f6f464f48d634ef3
I20260929 10:50:24.533197 27413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:24.524778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.20.80.195:45373)
0929 10:50:24.524941 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:24.525009 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:24.525459 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:24.525651 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:24.525654 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:24.526209 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:24.526395 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:24.526399 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.527056 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.527056 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:24.527815 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:24.527817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.528131 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.528131 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:24.528225 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:24.528763 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:24.528771 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:24.530150 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:24.531630 (+ 1480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:24.531636 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:24.531639 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:24.531856 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:24.532081 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:24.532082 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:24.532082 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:24.532185 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:24.532465 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:24.532472 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:24.532609 (+ 137us) client_negotiation.cc:769] Sending connection context
0929 10:50:24.532718 (+ 109us) client_negotiation.cc:240] Negotiation successful
0929 10:50:24.532871 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":100,"threads_started":1}
I20260929 10:50:24.533636 27411 heartbeater.cc:347] Connected to a master server at 127.20.80.254:44567
I20260929 10:50:24.533756 27411 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:24.533941 27411 heartbeater.cc:511] Master 127.20.80.254:44567 requested a full tablet report, sending...
I20260929 10:50:24.534327 26950 ts_manager.cc:194] Registered new tserver with Master: 658a8336746b42189751b2817f40a623 (127.20.80.195:35313)
I20260929 10:50:24.534813 26950 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.80.195@KRBTEST.COM'} at 127.20.80.195:45373
I20260929 10:50:24.544739 20803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.20.80.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.80.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.80.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.253.keytab.
Entry for principal oryx/127.20.80.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.253.keytab.
I20260929 10:50:24.567992 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.253:44859
--webserver_interface=127.20.80.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.253.keytab
--principal=oryx/127.20.80.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:40769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:44567,127.20.80.253:44859
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:24.648801 27416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:24.649230 27416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:24.649314 27416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:24.650493 27416 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:50:24.650561 27416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:24.650588 27416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:24.650614 27416 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:50:24.650635 27416 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:50:24.652113 27416 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:40769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:44567,127.20.80.253:44859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.253.keytab
--principal=oryx/127.20.80.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.253:44859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.80.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
W20260929 10:50:24.652259 27416 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.20.80.254:44567,127.20.80.253:44859'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260929 10:50:24.652424 27416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:24.652644 27416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:24.657586 27424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:24.657627 27421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:24.657569 27422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:24.657919 27416 server_base.cc:1034] running on GCE node
I20260929 10:50:24.658200 27416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:24.658517 27416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:24.659706 27416 hybrid_clock.cc:648] HybridClock initialized: now 1790679024659673 us; error 46 us; skew 500 ppm
Sep 29 10:50:24 dist-test-slave-vqch krb5kdc[26904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679024, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:24.662170 27416 init.cc:377] Logged in from keytab as oryx/127.20.80.253@KRBTEST.COM (short username oryx)
I20260929 10:50:24.662621 27416 webserver.cc:460] Webserver started at http://127.20.80.253:46375/ using document root <none> and password file <none>
I20260929 10:50:24.662868 27416 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:24.662914 27416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:24.663077 27416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:24.664098 27416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-1/data/instance:
uuid: "da5626be71c64349953ab4a5ecb1d8db"
format_stamp: "Formatted at 2026-09-29 10:50:24 on dist-test-slave-vqch"
server_key: "68ad1351074468fd4144b286f1c165c7"
server_key_iv: "3f3752e67d40bfd11906aa9152bca90a"
server_key_version: "encryptionkey@0"
I20260929 10:50:24.664435 27416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-1/wal/instance:
uuid: "da5626be71c64349953ab4a5ecb1d8db"
format_stamp: "Formatted at 2026-09-29 10:50:24 on dist-test-slave-vqch"
server_key: "68ad1351074468fd4144b286f1c165c7"
server_key_iv: "3f3752e67d40bfd11906aa9152bca90a"
server_key_version: "encryptionkey@0"
I20260929 10:50:24.665783 27416 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:50:24.666481 27431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:24.666656 27416 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:24.666759 27416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-1/wal
uuid: "da5626be71c64349953ab4a5ecb1d8db"
format_stamp: "Formatted at 2026-09-29 10:50:24 on dist-test-slave-vqch"
server_key: "68ad1351074468fd4144b286f1c165c7"
server_key_iv: "3f3752e67d40bfd11906aa9152bca90a"
server_key_version: "encryptionkey@0"
I20260929 10:50:24.666831 27416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:24.679404 27416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:24.682065 27416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:24.682235 27416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:24.686105 27416 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.253:44859
I20260929 10:50:24.686134 27485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.253:44859 every 8 connection(s)
I20260929 10:50:24.686476 27416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-1/data/info.pb
I20260929 10:50:24.687477 27486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:24.688937 27486 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:50:24.692390 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 27416
Sep 29 10:50:24 dist-test-slave-vqch krb5kdc[26904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679024, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.253@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM
Sep 29 10:50:24 dist-test-slave-vqch krb5kdc[26904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.80.253@KRBTEST.COM
I20260929 10:50:24.698669 26998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:24.689631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52444 (local address 127.20.80.254:44567)
0929 10:50:24.689701 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:24.689703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:24.689947 (+ 244us) server_negotiation.cc:407] Connection header received
0929 10:50:24.690473 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:24.690473 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:24.690483 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:24.690541 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:24.691133 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.691627 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.692850 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.693184 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.693253 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:24.695525 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:24.695527 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:24.695527 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:24.695575 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:24.697241 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:24.697610 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:24.697611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:24.697612 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:24.697675 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:24.697988 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:24.697989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:24.697989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:24.698134 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:24.698188 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:24.698496 (+ 308us) server_negotiation.cc:299] Negotiation successful
0929 10:50:24.698579 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 10:50:24.698846 27488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:24.689621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.0.0.1:52444)
0929 10:50:24.689818 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:24.689858 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:24.690380 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:24.690618 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:24.690621 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:24.690871 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:24.691056 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:24.691061 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.691704 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.691705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:24.692757 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:24.692760 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.693303 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.693320 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:24.693427 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:24.693962 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:24.693970 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:24.695438 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:24.697319 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:24.697324 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:24.697326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:24.697537 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:24.697762 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:24.697763 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:24.697763 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:24.697920 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:24.698204 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:24.698209 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:24.698358 (+ 149us) client_negotiation.cc:769] Sending connection context
0929 10:50:24.698471 (+ 113us) client_negotiation.cc:240] Negotiation successful
0929 10:50:24.698616 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":127,"threads_started":1}
I20260929 10:50:24.699767 27486 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:50:24.701885 27490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:24.692906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58020 (local address 127.20.80.253:44859)
0929 10:50:24.693075 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:24.693079 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:24.693176 (+ 97us) server_negotiation.cc:407] Connection header received
0929 10:50:24.693225 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:24.693228 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:24.693249 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:24.693336 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:24.694081 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.694803 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.695671 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.696086 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.696150 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:24.698852 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:24.698861 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:24.698861 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:24.698867 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:24.700671 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:24.701043 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:24.701046 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:24.701047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:24.701108 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:24.701314 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:24.701315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:24.701315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:24.701436 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:24.701604 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:24.701655 (+ 51us) server_negotiation.cc:299] Negotiation successful
0929 10:50:24.701779 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":100,"threads_started":1}
Sep 29 10:50:24 dist-test-slave-vqch krb5kdc[26904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679024, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.253@KRBTEST.COM for oryx/127.20.80.253@KRBTEST.COM
I20260929 10:50:24.707093 27488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:24.700040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:44859 (local address 127.0.0.1:58028)
0929 10:50:24.700083 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:24.700096 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:24.700124 (+ 28us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:24.700366 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:24.700367 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:24.700536 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:24.700574 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:24.700576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.701465 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.701466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:24.702101 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:24.702104 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.702505 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.702506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:24.702569 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:24.702867 (+ 298us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:24.702870 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:24.703884 (+ 1014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:24.706083 (+ 2199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:24.706084 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:24.706084 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:24.706258 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:24.706474 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:24.706474 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:24.706475 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:24.706528 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:24.706773 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:24.706774 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:24.706809 (+ 35us) client_negotiation.cc:769] Sending connection context
0929 10:50:24.706875 (+ 66us) client_negotiation.cc:240] Negotiation successful
0929 10:50:24.706969 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260929 10:50:24.707185 27491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:24.700091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58028 (local address 127.20.80.253:44859)
0929 10:50:24.700249 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:24.700250 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:24.700256 (+ 6us) server_negotiation.cc:407] Connection header received
0929 10:50:24.700270 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:24.700270 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:24.700277 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:24.700352 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:24.700654 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.701389 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.702181 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.702442 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.702506 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:24.703965 (+ 1459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:24.703966 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:24.703966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:24.703972 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:24.705969 (+ 1997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:24.706350 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:24.706351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:24.706351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:24.706409 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:24.706597 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:24.706598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:24.706599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:24.706707 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:24.706781 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:24.706907 (+ 126us) server_negotiation.cc:299] Negotiation successful
0929 10:50:24.707019 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":136,"thread_start_us":80,"threads_started":1}
I20260929 10:50:24.708469 27486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db: Bootstrap starting.
I20260929 10:50:24.709158 27486 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:24.709379 27486 log.cc:824] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:24.710047 27486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db: No bootstrap required, opened a new log
I20260929 10:50:24.711336 27486 raft_consensus.cc:348] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } }
I20260929 10:50:24.711454 27486 raft_consensus.cc:374] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:24.711480 27486 raft_consensus.cc:729] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da5626be71c64349953ab4a5ecb1d8db, State: Initialized, Role: FOLLOWER
I20260929 10:50:24.711566 27486 consensus_queue.cc:260] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } }
I20260929 10:50:24.711803 27493 sys_catalog.cc:455] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } } }
I20260929 10:50:24.711875 27486 sys_catalog.cc:564] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:50:24.711889 27493 sys_catalog.cc:458] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [sys.catalog]: This master's current role is: FOLLOWER
W20260929 10:50:24.713461 27504 catalog_manager.cc:1558] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:50:24.713526 27504 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:50:24.715348 27416 master_runner.cc:428] Detected that this master da5626be71c64349953ab4a5ecb1d8db is joining an existing cluster
I20260929 10:50:24.715387 27416 master_runner.cc:432] Initiating AddMaster RPC to add 127.20.80.253:44859
I20260929 10:50:24.723906 27491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:24.716705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58034 (local address 127.20.80.253:44859)
0929 10:50:24.716741 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:24.716743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:24.716840 (+ 97us) server_negotiation.cc:407] Connection header received
0929 10:50:24.716898 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:24.716899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:24.716908 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:24.717011 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:24.717365 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.718130 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.719196 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.719395 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.719488 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:24.720541 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:24.720542 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:24.720543 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:24.720548 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:24.722512 (+ 1964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:24.722869 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:24.722870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:24.722870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:24.722939 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:24.723222 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:24.723223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:24.723223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:24.723366 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:24.723442 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:24.723638 (+ 196us) server_negotiation.cc:299] Negotiation successful
0929 10:50:24.723777 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":19}
I20260929 10:50:24.723897 27511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:24.716662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:44859 (local address 127.0.0.1:58034)
0929 10:50:24.716775 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:24.716784 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:24.716843 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:24.717017 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:24.717017 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:24.717216 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:24.717272 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:24.717274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.718221 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.718222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:24.719089 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:24.719091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.719505 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.719506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:24.719562 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:24.720017 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:24.720022 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:24.720459 (+ 437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:24.722594 (+ 2135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:24.722596 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:24.722596 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:24.722807 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:24.723002 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:24.723002 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:24.723003 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:24.723127 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:24.723450 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:24.723451 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:24.723498 (+ 47us) client_negotiation.cc:769] Sending connection context
0929 10:50:24.723600 (+ 102us) client_negotiation.cc:240] Negotiation successful
0929 10:50:24.723711 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":63,"threads_started":1}
I20260929 10:50:24.725099 26998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:24.716484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52448 (local address 127.20.80.254:44567)
0929 10:50:24.716547 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:24.716549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:24.716665 (+ 116us) server_negotiation.cc:407] Connection header received
0929 10:50:24.716781 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:24.716782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:24.716794 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:24.716860 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:24.717426 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.718141 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.720123 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.720419 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.720494 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:24.722016 (+ 1522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:24.722017 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:24.722017 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:24.722024 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:24.723789 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:24.724295 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:24.724296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:24.724296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:24.724364 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:24.724585 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:24.724586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:24.724586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:24.724717 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:24.724821 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:24.724915 (+ 94us) server_negotiation.cc:299] Negotiation successful
0929 10:50:24.725010 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 10:50:24.725111 27510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:24.716418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.0.0.1:52448)
0929 10:50:24.716605 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:24.716616 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:24.716692 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:24.716908 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:24.716909 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:24.717266 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:24.717343 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:24.717346 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.719224 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.719224 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:24.720033 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:24.720036 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.720506 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.720507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:24.720572 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:24.721219 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:24.721224 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:24.721914 (+ 690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:24.723889 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:24.723890 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:24.723891 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:24.724114 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:24.724443 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:24.724443 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:24.724444 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:24.724503 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:24.724771 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:24.724772 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:24.724822 (+ 50us) client_negotiation.cc:769] Sending connection context
0929 10:50:24.724890 (+ 68us) client_negotiation.cc:240] Negotiation successful
0929 10:50:24.725012 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":80,"threads_started":1}
Sep 29 10:50:24 dist-test-slave-vqch krb5kdc[26904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679023, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.254@KRBTEST.COM for oryx/127.20.80.253@KRBTEST.COM
I20260929 10:50:24.734436 27491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:24.727181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58042 (local address 127.20.80.253:44859)
0929 10:50:24.727355 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:24.727357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:24.727365 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:50:24.727395 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:24.727396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:24.727405 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:24.727480 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:24.727908 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.728631 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.729458 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.729758 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.729827 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:24.731699 (+ 1872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:24.731700 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:24.731701 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:24.731707 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:24.733293 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:24.733647 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:24.733648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:24.733648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:24.733706 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:24.733893 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:24.733894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:24.733894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:24.734012 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:24.734079 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:24.734275 (+ 196us) server_negotiation.cc:299] Negotiation successful
0929 10:50:24.734350 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156}
I20260929 10:50:24.734477 27512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:24.727017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:44859 (local address 127.0.0.1:58042)
0929 10:50:24.727212 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:24.727253 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:24.727319 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:24.727524 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:24.727532 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:24.727773 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:24.727821 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:24.727823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.728722 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.728723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:24.729384 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:24.729387 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.729841 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.729842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:24.729912 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:24.730352 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:24.730356 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:24.731618 (+ 1262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:24.733385 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:24.733391 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:24.733393 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:24.733577 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:24.733774 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:24.733774 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:24.733775 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:24.733832 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:24.734106 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:24.734112 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:24.734157 (+ 45us) client_negotiation.cc:769] Sending connection context
0929 10:50:24.734234 (+ 77us) client_negotiation.cc:240] Negotiation successful
0929 10:50:24.734396 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":81,"threads_started":1}
I20260929 10:50:24.734994 27493 raft_consensus.cc:482] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:50:24.735092 27493 raft_consensus.cc:504] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } }
I20260929 10:50:24.735316 27493 leader_election.cc:288] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39326fe5440f4d74abd39e9b375c9eaf (127.20.80.254:44567)
I20260929 10:50:24.735555 26960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "da5626be71c64349953ab4a5ecb1d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39326fe5440f4d74abd39e9b375c9eaf" is_pre_election: true
I20260929 10:50:24.735688 26950 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.20.80.253:44859: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db"
member_type: NON_VOTER
last_known_addr {
host: "127.20.80.253"
port: 44859
}
attrs {
promote: true
}
}
dest_uuid: "39326fe5440f4d74abd39e9b375c9eaf"
cas_config_opid_index: -1
I20260929 10:50:24.735953 27432 leader_election.cc:302] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: da5626be71c64349953ab4a5ecb1d8db; no voters: 39326fe5440f4d74abd39e9b375c9eaf
I20260929 10:50:24.735972 26950 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: NON_VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: true } }
I20260929 10:50:24.736081 27493 raft_consensus.cc:2726] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260929 10:50:24.736431 27514 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER da5626be71c64349953ab4a5ecb1d8db (127.20.80.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: NON_VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: true } } }
I20260929 10:50:24.736704 27459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:24.737375 27514 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.20.80.253:44859
I20260929 10:50:24.737459 27514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "39326fe5440f4d74abd39e9b375c9eaf" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: NON_VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: true } } }
I20260929 10:50:24.737532 27514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:24.737596 27416 master.cc:537] Master@127.20.80.253:44859 shutting down...
I20260929 10:50:24.737627 27459 raft_consensus.cc:1261] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [term 1 FOLLOWER]: Refusing update from remote peer 39326fe5440f4d74abd39e9b375c9eaf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260929 10:50:24.738066 27513 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [LEADER]: Connected to new peer: Peer: permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: NON_VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 10:50:24.738739 26937 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260929 10:50:24.739259 26937 consensus_peers.cc:487] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf -> Peer da5626be71c64349953ab4a5ecb1d8db (127.20.80.253:44859): Couldn't send request to peer da5626be71c64349953ab4a5ecb1d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260929 10:50:24.740152 27416 raft_consensus.cc:2229] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:50:24.740231 27416 raft_consensus.cc:2258] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:24.740278 27416 tablet_replica.cc:330] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db: stopping tablet replica
I20260929 10:50:24.752949 27416 master.cc:552] Master@127.20.80.253:44859 shutdown complete.
I20260929 10:50:24.753051 27416 master_runner.cc:305] Clearing existing system tablet
I20260929 10:50:24.753209 27416 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:24.753260 27416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:24.753890 27416 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:50:24.754406 27518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:24.754518 27416 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:24.754572 27416 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:24.754601 27416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-1/wal
uuid: "da5626be71c64349953ab4a5ecb1d8db"
format_stamp: "Formatted at 2026-09-29 10:50:24 on dist-test-slave-vqch"
server_key: "68ad1351074468fd4144b286f1c165c7"
server_key_iv: "3f3752e67d40bfd11906aa9152bca90a"
server_key_version: "encryptionkey@0"
I20260929 10:50:24.755213 27416 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:24.756846 27416 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260929 10:50:24.756904 27416 log.cc:1196] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260929 10:50:24.757124 27416 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db: Deleting consensus metadata
I20260929 10:50:24.757251 27416 master_runner.cc:315] Copying system tablet from 127.20.80.254:44567
I20260929 10:50:24.757869 27416 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db: tablet copy: Beginning tablet copy session from remote peer at address 127.20.80.254:44567
I20260929 10:50:24.764161 26998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:24.758247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52450 (local address 127.20.80.254:44567)
0929 10:50:24.758303 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:24.758313 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:24.758357 (+ 44us) server_negotiation.cc:407] Connection header received
0929 10:50:24.758440 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:24.758441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:24.758452 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:24.758510 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:24.758850 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.759370 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.760097 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.760294 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.760384 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:24.761393 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:24.761394 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:24.761394 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:24.761401 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:24.762917 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:24.763341 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:24.763342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:24.763342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:24.763407 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:24.763619 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:24.763619 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:24.763620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:24.763734 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:24.763817 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:24.763988 (+ 171us) server_negotiation.cc:299] Negotiation successful
0929 10:50:24.764068 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260929 10:50:24.764176 27524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:24.758176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.0.0.1:52450)
0929 10:50:24.758290 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:24.758302 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:24.758375 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:24.758513 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:24.758514 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:24.758697 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:24.758749 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:24.758751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.759459 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.759460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:24.760011 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:24.760013 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.760386 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.760387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:24.760460 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:24.760919 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:24.760924 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:24.761285 (+ 361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:24.763041 (+ 1756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:24.763042 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:24.763043 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:24.763228 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:24.763475 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:24.763476 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:24.763477 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:24.763537 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:24.763828 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:24.763829 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:24.763874 (+ 45us) client_negotiation.cc:769] Sending connection context
0929 10:50:24.763962 (+ 88us) client_negotiation.cc:240] Negotiation successful
0929 10:50:24.764068 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":71,"threads_started":1}
I20260929 10:50:24.764535 26970 tablet_copy_service.cc:140] P 39326fe5440f4d74abd39e9b375c9eaf: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer da5626be71c64349953ab4a5ecb1d8db ({username='oryx', principal='oryx/127.20.80.253@KRBTEST.COM'} at 127.0.0.1:52450)
I20260929 10:50:24.764623 26970 tablet_copy_service.cc:161] P 39326fe5440f4d74abd39e9b375c9eaf: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer da5626be71c64349953ab4a5ecb1d8db at {username='oryx', principal='oryx/127.20.80.253@KRBTEST.COM'} at 127.0.0.1:52450: session id = da5626be71c64349953ab4a5ecb1d8db-00000000000000000000000000000000
I20260929 10:50:24.765458 26970 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf: Tablet Copy: opened 0 blocks and 1 log segments
I20260929 10:50:24.766116 27416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260929 10:50:24.767369 27416 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db: tablet copy: Starting download of 0 data blocks...
I20260929 10:50:24.767519 27416 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db: tablet copy: Starting download of 1 WAL segments...
I20260929 10:50:24.768368 27416 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260929 10:50:24.769290 26970 tablet_copy_service.cc:342] P 39326fe5440f4d74abd39e9b375c9eaf: Request end of tablet copy session da5626be71c64349953ab4a5ecb1d8db-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.20.80.253@KRBTEST.COM'} at 127.0.0.1:52450
I20260929 10:50:24.769353 26970 tablet_copy_service.cc:433] P 39326fe5440f4d74abd39e9b375c9eaf: ending tablet copy session da5626be71c64349953ab4a5ecb1d8db-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer da5626be71c64349953ab4a5ecb1d8db
I20260929 10:50:24.771548 27416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:24.772873 27525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:24.772949 27528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:24.773084 27526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:24.773126 27416 server_base.cc:1034] running on GCE node
I20260929 10:50:24.773254 27416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:24.773394 27416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:24.774544 27416 hybrid_clock.cc:648] HybridClock initialized: now 1790679024774537 us; error 34 us; skew 500 ppm
Sep 29 10:50:24 dist-test-slave-vqch krb5kdc[26904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679024, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:24.775887 27416 init.cc:377] Logged in from keytab as oryx/127.20.80.253@KRBTEST.COM (short username oryx)
I20260929 10:50:24.776154 27416 webserver.cc:460] Webserver started at http://127.20.80.253:33697/ using document root <none> and password file <none>
I20260929 10:50:24.776322 27416 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:24.776372 27416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:24.777189 27416 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:50:24.777840 27536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:24.778014 27416 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:50:24.778070 27416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-1/wal
uuid: "da5626be71c64349953ab4a5ecb1d8db"
format_stamp: "Formatted at 2026-09-29 10:50:24 on dist-test-slave-vqch"
server_key: "68ad1351074468fd4144b286f1c165c7"
server_key_iv: "3f3752e67d40bfd11906aa9152bca90a"
server_key_version: "encryptionkey@0"
I20260929 10:50:24.778131 27416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:24.806180 27416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:24.807994 27416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:24.811149 27416 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.253:44859
I20260929 10:50:24.811177 27588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.253:44859 every 8 connection(s)
I20260929 10:50:24.811506 27416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-1/data/info.pb
I20260929 10:50:24.812227 27589 sys_catalog.cc:263] Verifying existing consensus state
I20260929 10:50:24.812704 27589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db: Bootstrap starting.
I20260929 10:50:24.815021 27589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 10:50:24.815234 27589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db: Bootstrap complete.
I20260929 10:50:24.815430 27589 raft_consensus.cc:348] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: NON_VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: true } }
I20260929 10:50:24.815518 27589 raft_consensus.cc:729] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da5626be71c64349953ab4a5ecb1d8db, State: Initialized, Role: LEARNER
I20260929 10:50:24.815569 27589 consensus_queue.cc:260] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: NON_VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: true } }
I20260929 10:50:24.815797 27589 sys_catalog.cc:564] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:50:24.815773 27592 sys_catalog.cc:455] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: NON_VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: true } } }
I20260929 10:50:24.815846 27592 sys_catalog.cc:458] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [sys.catalog]: This master's current role is: LEARNER
I20260929 10:50:24.817198 27603 catalog_manager.cc:1259] Loaded cluster ID: 3fef4c4dad1e479d988b0cb330c76f15
I20260929 10:50:24.817260 27603 catalog_manager.cc:1552] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db: loading cluster ID for follower catalog manager: success
I20260929 10:50:24.818426 27603 catalog_manager.cc:1574] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db: acquiring CA information for follower catalog manager: success
I20260929 10:50:24.818706 27603 catalog_manager.cc:1602] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 10:50:24.860531 27604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:24.854200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58076 (local address 127.20.80.253:44859)
0929 10:50:24.854353 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:24.854355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:24.854362 (+ 7us) server_negotiation.cc:407] Connection header received
0929 10:50:24.854385 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:24.854385 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:24.854394 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:24.854476 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:24.855030 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.855688 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.856424 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:24.856624 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:24.856712 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:24.858021 (+ 1309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:24.858022 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:24.858023 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:24.858031 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:24.859382 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:24.859729 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:24.859730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:24.859730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:24.859792 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:24.859998 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:24.859999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:24.859999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:24.860113 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:24.860219 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:24.860336 (+ 117us) server_negotiation.cc:299] Negotiation successful
0929 10:50:24.860419 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":75,"threads_started":1}
WARNING: no policy specified for oryx/127.20.80.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.80.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.80.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.252.keytab.
Entry for principal oryx/127.20.80.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.252.keytab.
I20260929 10:50:24.885684 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.252:39713
--webserver_interface=127.20.80.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.252.keytab
--principal=oryx/127.20.80.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:40769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:44567,127.20.80.253:44859,127.20.80.252:39713
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:24.965409 27607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:24.965616 27607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:24.965638 27607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:24.966985 27607 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:50:24.967026 27607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:24.967041 27607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:24.967052 27607 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:50:24.967064 27607 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:50:24.968475 27607 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:40769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:44567,127.20.80.253:44859,127.20.80.252:39713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.252.keytab
--principal=oryx/127.20.80.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.252:39713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.80.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:24.968655 27607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:24.968818 27607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:24.973543 27614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:24.973557 27616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:24.973557 27613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:24.973635 27607 server_base.cc:1034] running on GCE node
I20260929 10:50:24.974017 27607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:24.974282 27607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:24.975445 27607 hybrid_clock.cc:648] HybridClock initialized: now 1790679024975435 us; error 54 us; skew 500 ppm
Sep 29 10:50:24 dist-test-slave-vqch krb5kdc[26904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679024, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:24.978073 27607 init.cc:377] Logged in from keytab as oryx/127.20.80.252@KRBTEST.COM (short username oryx)
I20260929 10:50:24.978541 27607 webserver.cc:460] Webserver started at http://127.20.80.252:41449/ using document root <none> and password file <none>
I20260929 10:50:24.978778 27607 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:24.978845 27607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:24.979010 27607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:24.980087 27607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-2/data/instance:
uuid: "8d37f646d63549d480f2c2e479a0826c"
format_stamp: "Formatted at 2026-09-29 10:50:24 on dist-test-slave-vqch"
server_key: "65b081e03d2aa53e64d8306b459766a0"
server_key_iv: "4a83c27c53cfd7b21438aebdf7672e91"
server_key_version: "encryptionkey@0"
I20260929 10:50:24.980443 27607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-2/wal/instance:
uuid: "8d37f646d63549d480f2c2e479a0826c"
format_stamp: "Formatted at 2026-09-29 10:50:24 on dist-test-slave-vqch"
server_key: "65b081e03d2aa53e64d8306b459766a0"
server_key_iv: "4a83c27c53cfd7b21438aebdf7672e91"
server_key_version: "encryptionkey@0"
I20260929 10:50:24.981823 27607 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 10:50:24.982578 27623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:24.982762 27607 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:50:24.982883 27607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-2/wal
uuid: "8d37f646d63549d480f2c2e479a0826c"
format_stamp: "Formatted at 2026-09-29 10:50:24 on dist-test-slave-vqch"
server_key: "65b081e03d2aa53e64d8306b459766a0"
server_key_iv: "4a83c27c53cfd7b21438aebdf7672e91"
server_key_version: "encryptionkey@0"
I20260929 10:50:24.982960 27607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:24.997603 27607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:24.999878 27607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:25.000034 27607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:25.004004 27607 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.252:39713
I20260929 10:50:25.004040 27675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.252:39713 every 8 connection(s)
I20260929 10:50:25.004412 27607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-2/data/info.pb
I20260929 10:50:25.005450 27676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:25.006963 27676 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:50:25.010264 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 27607
Sep 29 10:50:25 dist-test-slave-vqch krb5kdc[26904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679024, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.252@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM
Sep 29 10:50:25 dist-test-slave-vqch krb5kdc[26904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.80.252@KRBTEST.COM
I20260929 10:50:25.017350 26998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.007769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52458 (local address 127.20.80.254:44567)
0929 10:50:25.007844 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:25.007846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:25.007934 (+ 88us) server_negotiation.cc:407] Connection header received
0929 10:50:25.008473 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:25.008474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:25.008486 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:25.008540 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:25.009203 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.009783 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.011019 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.011216 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.011302 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.013809 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:25.013811 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:25.013811 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:25.013819 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:25.015771 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.016240 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.016242 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.016242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.016302 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.016594 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.016595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.016595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.016697 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:25.016778 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:25.017138 (+ 360us) server_negotiation.cc:299] Negotiation successful
0929 10:50:25.017278 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 10:50:25.017503 27678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.007621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.0.0.1:52458)
0929 10:50:25.007778 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:25.007846 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:25.008371 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:25.008609 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:25.008613 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:25.008854 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:25.009105 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.009110 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.009907 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.009908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.010919 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.010922 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.011342 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.011343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.011462 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.012014 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:25.012024 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:25.013717 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:25.015861 (+ 2144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:25.015868 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:25.015870 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:25.016151 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:25.016379 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:25.016380 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:25.016380 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:25.016522 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:25.016805 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:25.016812 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:25.016996 (+ 184us) client_negotiation.cc:769] Sending connection context
0929 10:50:25.017102 (+ 106us) client_negotiation.cc:240] Negotiation successful
0929 10:50:25.017292 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":94,"threads_started":1}
I20260929 10:50:25.018136 27676 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:50:25.019994 27679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.010757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35800 (local address 127.20.80.252:39713)
0929 10:50:25.010925 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:25.010930 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:25.010954 (+ 24us) server_negotiation.cc:407] Connection header received
0929 10:50:25.010971 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:25.010973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:25.010988 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:25.011069 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:25.011641 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.012375 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.013371 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.013813 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.013898 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.016758 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:25.016765 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:25.016765 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:25.016774 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:25.018801 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.019158 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.019161 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.019161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.019221 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.019429 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.019430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.019430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.019564 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:25.019703 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:25.019789 (+ 86us) server_negotiation.cc:299] Negotiation successful
0929 10:50:25.019872 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":89,"threads_started":1}
Sep 29 10:50:25 dist-test-slave-vqch krb5kdc[26904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679024, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.252@KRBTEST.COM for oryx/127.20.80.253@KRBTEST.COM
I20260929 10:50:25.024287 27678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.018402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:44859 (local address 127.0.0.1:58080)
0929 10:50:25.018446 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:25.018457 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:25.018491 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:25.018698 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:25.018698 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:25.018868 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:25.018912 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.018913 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.019561 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.019562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.020100 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.020102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.020538 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.020538 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.020608 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.020913 (+ 305us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:25.020917 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:25.022107 (+ 1190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:25.023527 (+ 1420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:25.023528 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:25.023528 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:25.023677 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:25.023866 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:25.023866 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:25.023867 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:25.023914 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:25.024085 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:25.024086 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:25.024117 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 10:50:25.024170 (+ 53us) client_negotiation.cc:240] Negotiation successful
0929 10:50:25.024232 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260929 10:50:25.024312 27604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.018508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58080 (local address 127.20.80.253:44859)
0929 10:50:25.018575 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:25.018578 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:25.018586 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:50:25.018612 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:25.018613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:25.018624 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:25.018694 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:25.018993 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.019472 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.020182 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.020444 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.020530 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.022178 (+ 1648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:25.022179 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:25.022180 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:25.022186 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:25.023460 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.023737 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.023738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.023739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.023782 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.023950 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.023950 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.023951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.024038 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:25.024080 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:25.024186 (+ 106us) server_negotiation.cc:299] Negotiation successful
0929 10:50:25.024251 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 10:50:25.024676 27676 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.80.252" port: 39713 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 10:50:25 dist-test-slave-vqch krb5kdc[26904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679024, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.252@KRBTEST.COM for oryx/127.20.80.252@KRBTEST.COM
I20260929 10:50:25.030256 27678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.024915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:39713 (local address 127.0.0.1:35814)
0929 10:50:25.024963 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:25.024973 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:25.025029 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:25.025132 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:25.025132 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:25.025231 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:25.025256 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.025257 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.025810 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.025811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.026246 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.026247 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.026560 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.026561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.026595 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.026864 (+ 269us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:25.026867 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:25.027991 (+ 1124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:25.029480 (+ 1489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:25.029481 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:25.029481 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:25.029636 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:25.029794 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:25.029795 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:25.029795 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:25.029861 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:25.030050 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:25.030051 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:25.030081 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 10:50:25.030134 (+ 53us) client_negotiation.cc:240] Negotiation successful
0929 10:50:25.030201 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260929 10:50:25.030402 27679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.024966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35814 (local address 127.20.80.252:39713)
0929 10:50:25.025033 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:25.025035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:25.025064 (+ 29us) server_negotiation.cc:407] Connection header received
0929 10:50:25.025087 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:25.025087 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:25.025095 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:25.025130 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:25.025310 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.025748 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.026306 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.026502 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.026549 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.028091 (+ 1542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:25.028092 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:25.028092 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:25.028100 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:25.029413 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.029697 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.029698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.029698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.029741 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.029914 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.029914 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.029915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.030000 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:25.030046 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:25.030161 (+ 115us) server_negotiation.cc:299] Negotiation successful
0929 10:50:25.030243 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":51}
I20260929 10:50:25.031701 27676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c: Bootstrap starting.
I20260929 10:50:25.032457 27676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:25.032687 27676 log.cc:824] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:25.033407 27676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c: No bootstrap required, opened a new log
I20260929 10:50:25.034814 27676 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 39713 } }
I20260929 10:50:25.034962 27676 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:25.035009 27676 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d37f646d63549d480f2c2e479a0826c, State: Initialized, Role: FOLLOWER
I20260929 10:50:25.035156 27676 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 39713 } }
I20260929 10:50:25.035414 27681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 39713 } } }
I20260929 10:50:25.035471 27676 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:50:25.035491 27681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [sys.catalog]: This master's current role is: FOLLOWER
W20260929 10:50:25.037053 27692 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:50:25.037108 27692 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:50:25.039283 27607 master_runner.cc:428] Detected that this master 8d37f646d63549d480f2c2e479a0826c is joining an existing cluster
I20260929 10:50:25.039338 27607 master_runner.cc:432] Initiating AddMaster RPC to add 127.20.80.252:39713
I20260929 10:50:25.047415 27679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.040778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35818 (local address 127.20.80.252:39713)
0929 10:50:25.040814 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:25.040816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:25.040936 (+ 120us) server_negotiation.cc:407] Connection header received
0929 10:50:25.041049 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:25.041050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:25.041059 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:25.041084 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:25.041460 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.041909 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.042776 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.043045 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.043078 (+ 33us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.044080 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:25.044081 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:25.044082 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:25.044088 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:25.045659 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.046553 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.046554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.046555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.046605 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.046802 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.046802 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.046803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.046974 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:25.047057 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:25.047220 (+ 163us) server_negotiation.cc:299] Negotiation successful
0929 10:50:25.047334 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":18}
I20260929 10:50:25.047421 27699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.040717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:39713 (local address 127.0.0.1:35818)
0929 10:50:25.040883 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:25.040892 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:25.040954 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:25.041133 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:25.041134 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:25.041313 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:25.041379 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.041380 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.041966 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.041967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.042693 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.042695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.043141 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.043141 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.043200 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.043707 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:25.043711 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:25.043996 (+ 285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:25.046293 (+ 2297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:25.046294 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:25.046294 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:25.046466 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:25.046678 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:25.046678 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:25.046679 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:25.046736 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:25.047048 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:25.047049 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:25.047105 (+ 56us) client_negotiation.cc:769] Sending connection context
0929 10:50:25.047184 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 10:50:25.047285 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":79,"threads_started":1}
I20260929 10:50:25.051436 27604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.041050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58096 (local address 127.20.80.253:44859)
0929 10:50:25.041089 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:25.041091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:25.045130 (+ 4039us) server_negotiation.cc:407] Connection header received
0929 10:50:25.045181 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:25.045182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:25.045196 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:25.045265 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:25.045733 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.046219 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.047335 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.047676 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.047718 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.048918 (+ 1200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:25.048920 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:25.048920 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:25.048926 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:25.050395 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.050774 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.050775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.050775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.050830 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.051025 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.051025 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.051025 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.051128 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:25.051182 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:25.051315 (+ 133us) server_negotiation.cc:299] Negotiation successful
0929 10:50:25.051376 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 10:50:25.051503 27700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.041138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:44859 (local address 127.0.0.1:58096)
0929 10:50:25.045050 (+ 3912us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:25.045061 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:25.045122 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:25.045271 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:25.045272 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:25.045578 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:25.045646 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.045649 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.046295 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.046296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.047210 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.047212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.047844 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.047845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.047919 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.048444 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:25.048449 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:25.048840 (+ 391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:25.050472 (+ 1632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:25.050473 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:25.050473 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:25.050681 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:25.050903 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:25.050904 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:25.050905 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:25.050966 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:25.051184 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:25.051185 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:25.051226 (+ 41us) client_negotiation.cc:769] Sending connection context
0929 10:50:25.051301 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 10:50:25.051395 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3895,"thread_start_us":66,"threads_started":1}
I20260929 10:50:25.064505 26998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.040859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52466 (local address 127.20.80.254:44567)
0929 10:50:25.040914 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:25.040916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:25.040923 (+ 7us) server_negotiation.cc:407] Connection header received
0929 10:50:25.040949 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:25.040951 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:25.040961 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:25.041032 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:25.041466 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.042282 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.043300 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.044002 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.044071 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.061996 (+ 17925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:25.061998 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:25.061998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:25.062006 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:25.063435 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.063820 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.063821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.063821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.063872 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.064059 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.064060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.064060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.064164 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:25.064212 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:25.064379 (+ 167us) server_negotiation.cc:299] Negotiation successful
0929 10:50:25.064440 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 10:50:25.064625 27698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.040508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.0.0.1:52466)
0929 10:50:25.040657 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:25.040667 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:25.040735 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:25.041077 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:25.041078 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:25.041312 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:25.041378 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.041380 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.042372 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.042373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.043214 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.043216 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.044124 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.044125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.044195 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.061441 (+ 17246us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:25.061448 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:25.061874 (+ 426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:25.063523 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:25.063524 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:25.063525 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:25.063745 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:25.063933 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:25.063934 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:25.063935 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:25.063999 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:25.064222 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:25.064223 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:25.064272 (+ 49us) client_negotiation.cc:769] Sending connection context
0929 10:50:25.064346 (+ 74us) client_negotiation.cc:240] Negotiation successful
0929 10:50:25.064467 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":77,"threads_started":1}
Sep 29 10:50:25 dist-test-slave-vqch krb5kdc[26904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679023, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.254@KRBTEST.COM for oryx/127.20.80.252@KRBTEST.COM
I20260929 10:50:25.071156 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 27012
I20260929 10:50:25.074470 27512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.066614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:39713 (local address 127.0.0.1:35828)
0929 10:50:25.066663 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:25.066672 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:25.066711 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:25.066846 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:25.066847 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:25.066995 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:25.067032 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.067034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.067639 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.067640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.068171 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.068173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.068546 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.068547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.068613 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.069028 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:25.069033 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:25.070502 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:25.073371 (+ 2869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:25.073372 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:25.073372 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:25.073577 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:25.073794 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:25.073795 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:25.073795 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:25.073864 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:25.074146 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:25.074147 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:25.074195 (+ 48us) client_negotiation.cc:769] Sending connection context
0929 10:50:25.074267 (+ 72us) client_negotiation.cc:240] Negotiation successful
0929 10:50:25.074369 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260929 10:50:25.074527 27679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.066676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35828 (local address 127.20.80.252:39713)
0929 10:50:25.066736 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:25.066738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:25.066747 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:50:25.066779 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:25.066780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:25.066789 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:25.066833 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:25.067108 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.067561 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.068253 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.068464 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.068545 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.070664 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:25.070666 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:25.070666 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:25.070673 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:25.073278 (+ 2605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.073654 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.073655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.073655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.073717 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.073932 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.073933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.073933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.074076 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:25.074144 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:25.074306 (+ 162us) server_negotiation.cc:299] Negotiation successful
0929 10:50:25.074393 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 10:50:25.075007 26950 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.20.80.252:39713: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "8d37f646d63549d480f2c2e479a0826c"
member_type: NON_VOTER
last_known_addr {
host: "127.20.80.252"
port: 39713
}
attrs {
promote: true
}
}
dest_uuid: "39326fe5440f4d74abd39e9b375c9eaf"
cas_config_opid_index: 5
I20260929 10:50:25.075241 26950 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: NON_VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: true } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: NON_VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: true } }
I20260929 10:50:25.075709 27514 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 8d37f646d63549d480f2c2e479a0826c (127.20.80.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: NON_VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: true } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: NON_VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: true } } }
I20260929 10:50:25.076676 27514 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.20.80.252:39713
I20260929 10:50:25.076738 27514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "39326fe5440f4d74abd39e9b375c9eaf" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: NON_VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: true } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: NON_VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: true } } }
I20260929 10:50:25.076799 27514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:25.077184 27650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:25.077399 27607 master.cc:537] Master@127.20.80.252:39713 shutting down...
I20260929 10:50:25.078312 27650 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [term 1 FOLLOWER]: Refusing update from remote peer 39326fe5440f4d74abd39e9b375c9eaf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260929 10:50:25.078622 27513 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: NON_VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 10:50:25.079288 26936 consensus_peers.cc:487] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf -> Peer 8d37f646d63549d480f2c2e479a0826c (127.20.80.252:39713): Couldn't send request to peer 8d37f646d63549d480f2c2e479a0826c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260929 10:50:25.080660 27512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.075671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:44859 (local address 127.0.0.1:58108)
0929 10:50:25.077079 (+ 1408us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:25.077089 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:25.077131 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:25.077353 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:25.077354 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:25.077355 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:25.077392 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.077393 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.078305 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.078306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.079498 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.079501 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.080422 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.080423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.080489 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.080508 (+ 19us) client_negotiation.cc:769] Sending connection context
0929 10:50:25.080557 (+ 49us) client_negotiation.cc:240] Negotiation successful
0929 10:50:25.080593 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1393}
I20260929 10:50:25.080807 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.193:41319
--local_ip_for_outbound_sockets=127.20.80.193
--tserver_master_addrs=127.20.80.254:44567
--webserver_port=34809
--webserver_interface=127.20.80.193
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.193.keytab
--principal=oryx/127.20.80.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:40769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.20.80.254:44567,127.20.80.253:44859
--tserver_master_addrs=127.20.80.254:44567,127.20.80.253:44859,127.20.80.252:39713 with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:50:25.080904 27604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.075750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58108 (local address 127.20.80.253:44859)
0929 10:50:25.075792 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:25.075794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:25.077187 (+ 1393us) server_negotiation.cc:407] Connection header received
0929 10:50:25.077219 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:25.077219 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:25.077243 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:25.077297 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:25.077464 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.078211 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.079595 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.080185 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.080312 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.080388 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:25.080791 (+ 403us) server_negotiation.cc:299] Negotiation successful
0929 10:50:25.080841 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 10:50:25.081148 27563 raft_consensus.cc:1261] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [term 1 LEARNER]: Refusing update from remote peer 39326fe5440f4d74abd39e9b375c9eaf: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260929 10:50:25.081369 27513 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [LEADER]: Connected to new peer: Peer: permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: NON_VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260929 10:50:25.081910 27563 raft_consensus.cc:2932] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 8d37f646d63549d480f2c2e479a0826c (127.20.80.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: NON_VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: true } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: NON_VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: true } } }
I20260929 10:50:25.083244 27592 sys_catalog.cc:455] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39326fe5440f4d74abd39e9b375c9eaf. Latest consensus state: current_term: 1 leader_uuid: "39326fe5440f4d74abd39e9b375c9eaf" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: NON_VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: true } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: NON_VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: true } } }
I20260929 10:50:25.083346 27592 sys_catalog.cc:458] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [sys.catalog]: This master's current role is: LEARNER
I20260929 10:50:25.083402 27592 sys_catalog.cc:455] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "39326fe5440f4d74abd39e9b375c9eaf" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: NON_VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: true } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: NON_VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: true } } }
I20260929 10:50:25.083459 27592 sys_catalog.cc:458] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [sys.catalog]: This master's current role is: LEARNER
I20260929 10:50:25.083482 27607 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:50:25.083563 27607 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:25.083609 27607 tablet_replica.cc:330] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c: stopping tablet replica
I20260929 10:50:25.083768 27513 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf: attempting to promote NON_VOTER da5626be71c64349953ab4a5ecb1d8db to VOTER
I20260929 10:50:25.083973 27513 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: false } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: NON_VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: true } }
I20260929 10:50:25.085121 27512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.085026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:39713 (local address 127.0.0.1:35830)
0929 10:50:25.085061 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:25.085096 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.252:39713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260929 10:50:25.085183 26936 consensus_peers.cc:487] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf -> Peer 8d37f646d63549d480f2c2e479a0826c (127.20.80.252:39713): Couldn't send request to peer 8d37f646d63549d480f2c2e479a0826c. Status: Network error: Client connection negotiation failed: client connection to 127.20.80.252:39713: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260929 10:50:25.085269 27563 raft_consensus.cc:1261] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [term 1 LEARNER]: Refusing update from remote peer 39326fe5440f4d74abd39e9b375c9eaf: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260929 10:50:25.085430 27514 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [LEADER]: Connected to new peer: Peer: permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260929 10:50:25.085983 27514 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, da5626be71c64349953ab4a5ecb1d8db (127.20.80.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: false } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: NON_VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: true } } }
I20260929 10:50:25.086863 27514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "39326fe5440f4d74abd39e9b375c9eaf" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: false } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: NON_VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: true } } }
I20260929 10:50:25.086939 27514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:25.087174 27513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "39326fe5440f4d74abd39e9b375c9eaf" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: false } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: NON_VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: true } } }
I20260929 10:50:25.087237 27513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:25.087462 27563 raft_consensus.cc:2932] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, da5626be71c64349953ab4a5ecb1d8db (127.20.80.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: false } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: NON_VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: true } } }
I20260929 10:50:25.088300 27592 sys_catalog.cc:455] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "39326fe5440f4d74abd39e9b375c9eaf" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: false } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: NON_VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: true } } }
I20260929 10:50:25.088398 27592 sys_catalog.cc:458] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:50:25.096447 27607 master.cc:552] Master@127.20.80.252:39713 shutdown complete.
I20260929 10:50:25.096498 27607 master_runner.cc:305] Clearing existing system tablet
I20260929 10:50:25.096627 27607 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:25.096676 27607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:25.097733 27607 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 10:50:25.098276 27707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:25.098390 27607 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:25.098438 27607 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:25.098461 27607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-2/wal
uuid: "8d37f646d63549d480f2c2e479a0826c"
format_stamp: "Formatted at 2026-09-29 10:50:24 on dist-test-slave-vqch"
server_key: "65b081e03d2aa53e64d8306b459766a0"
server_key_iv: "4a83c27c53cfd7b21438aebdf7672e91"
server_key_version: "encryptionkey@0"
I20260929 10:50:25.099261 27607 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:25.100741 27607 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260929 10:50:25.100816 27607 log.cc:1196] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260929 10:50:25.101048 27607 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c: Deleting consensus metadata
I20260929 10:50:25.101169 27607 master_runner.cc:315] Copying system tablet from 127.20.80.254:44567
I20260929 10:50:25.101749 27607 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c: tablet copy: Beginning tablet copy session from remote peer at address 127.20.80.254:44567
I20260929 10:50:25.109053 26998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.102117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52470 (local address 127.20.80.254:44567)
0929 10:50:25.102159 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:25.102161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:25.102225 (+ 64us) server_negotiation.cc:407] Connection header received
0929 10:50:25.102312 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:25.102314 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:25.102324 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:25.102379 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:25.102780 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.103585 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.104569 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.104884 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.104955 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.106069 (+ 1114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:25.106071 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:25.106071 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:25.106078 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:25.107737 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.108144 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.108146 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.108146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.108211 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.108451 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.108452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.108452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.108623 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:25.108704 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:25.108863 (+ 159us) server_negotiation.cc:299] Negotiation successful
0929 10:50:25.108941 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 10:50:25.109064 27713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.102046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.0.0.1:52470)
0929 10:50:25.102170 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:25.102180 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:25.102238 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:25.102421 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:25.102422 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:25.102620 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:25.102691 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.102694 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.103671 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.103673 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.104476 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.104478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.104963 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.104964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.105089 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.105595 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:25.105600 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:25.105980 (+ 380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:25.107836 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:25.107837 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:25.107837 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:25.108064 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:25.108319 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:25.108320 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:25.108320 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:25.108385 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:25.108698 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:25.108699 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:25.108746 (+ 47us) client_negotiation.cc:769] Sending connection context
0929 10:50:25.108818 (+ 72us) client_negotiation.cc:240] Negotiation successful
0929 10:50:25.108927 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":71,"threads_started":1}
I20260929 10:50:25.109634 26970 tablet_copy_service.cc:140] P 39326fe5440f4d74abd39e9b375c9eaf: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 8d37f646d63549d480f2c2e479a0826c ({username='oryx', principal='oryx/127.20.80.252@KRBTEST.COM'} at 127.0.0.1:52470)
I20260929 10:50:25.109692 26970 tablet_copy_service.cc:161] P 39326fe5440f4d74abd39e9b375c9eaf: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 8d37f646d63549d480f2c2e479a0826c at {username='oryx', principal='oryx/127.20.80.252@KRBTEST.COM'} at 127.0.0.1:52470: session id = 8d37f646d63549d480f2c2e479a0826c-00000000000000000000000000000000
I20260929 10:50:25.110260 26970 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf: Tablet Copy: opened 0 blocks and 1 log segments
I20260929 10:50:25.110723 27607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260929 10:50:25.111965 27607 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c: tablet copy: Starting download of 0 data blocks...
I20260929 10:50:25.112134 27607 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c: tablet copy: Starting download of 1 WAL segments...
I20260929 10:50:25.112924 27607 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260929 10:50:25.113831 26970 tablet_copy_service.cc:342] P 39326fe5440f4d74abd39e9b375c9eaf: Request end of tablet copy session 8d37f646d63549d480f2c2e479a0826c-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.20.80.252@KRBTEST.COM'} at 127.0.0.1:52470
I20260929 10:50:25.113883 26970 tablet_copy_service.cc:433] P 39326fe5440f4d74abd39e9b375c9eaf: ending tablet copy session 8d37f646d63549d480f2c2e479a0826c-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 8d37f646d63549d480f2c2e479a0826c
I20260929 10:50:25.116824 27607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:25.118160 27714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:25.121923 27715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:25.122097 27717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:25.122507 27607 server_base.cc:1034] running on GCE node
I20260929 10:50:25.122593 27607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:25.122718 27607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:25.123858 27607 hybrid_clock.cc:648] HybridClock initialized: now 1790679025123849 us; error 51 us; skew 500 ppm
Sep 29 10:50:25 dist-test-slave-vqch krb5kdc[26904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679025, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:25.125437 27607 init.cc:377] Logged in from keytab as oryx/127.20.80.252@KRBTEST.COM (short username oryx)
I20260929 10:50:25.125692 27607 webserver.cc:460] Webserver started at http://127.20.80.252:44993/ using document root <none> and password file <none>
I20260929 10:50:25.125823 27607 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:25.125861 27607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:25.126600 27607 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:25.127219 27724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:25.127368 27607 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:25.127446 27607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-2/wal
uuid: "8d37f646d63549d480f2c2e479a0826c"
format_stamp: "Formatted at 2026-09-29 10:50:24 on dist-test-slave-vqch"
server_key: "65b081e03d2aa53e64d8306b459766a0"
server_key_iv: "4a83c27c53cfd7b21438aebdf7672e91"
server_key_version: "encryptionkey@0"
I20260929 10:50:25.127544 27607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:25.145561 27607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:25.147643 27607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:25.151207 27607 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.252:39713
I20260929 10:50:25.151328 27777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.252:39713 every 8 connection(s)
I20260929 10:50:25.151731 27607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/master-2/data/info.pb
I20260929 10:50:25.152791 27778 sys_catalog.cc:263] Verifying existing consensus state
I20260929 10:50:25.153349 27778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c: Bootstrap starting.
I20260929 10:50:25.156559 27778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 10:50:25.156800 27778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c: Bootstrap complete.
I20260929 10:50:25.157032 27778 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: false } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: NON_VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: true } }
I20260929 10:50:25.157133 27778 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8d37f646d63549d480f2c2e479a0826c, State: Initialized, Role: LEARNER
I20260929 10:50:25.157176 27778 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: false } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: NON_VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: true } }
I20260929 10:50:25.157385 27778 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:50:25.157379 27781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: false } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: NON_VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: true } } }
I20260929 10:50:25.157625 27781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [sys.catalog]: This master's current role is: LEARNER
I20260929 10:50:25.158962 27792 catalog_manager.cc:1259] Loaded cluster ID: 3fef4c4dad1e479d988b0cb330c76f15
I20260929 10:50:25.159870 27792 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c: loading cluster ID for follower catalog manager: success
I20260929 10:50:25.161068 27792 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c: acquiring CA information for follower catalog manager: success
I20260929 10:50:25.161384 27792 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260929 10:50:25.175045 27702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:25.175310 27702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:25.175361 27702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:25.176506 27702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:25.176566 27702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:25.176610 27702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.193
I20260929 10:50:25.178213 27702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:40769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.193.keytab
--principal=oryx/127.20.80.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.193:41319
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.80.193
--webserver_port=34809
--tserver_master_addrs=127.20.80.254:44567,127.20.80.253:44859,127.20.80.252:39713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.193
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:25.178470 27702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:25.178681 27702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:25.183589 27798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:25.183594 27797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:25.183825 27800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:25.184018 27702 server_base.cc:1034] running on GCE node
I20260929 10:50:25.184203 27702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:25.184388 27702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:25.185519 27702 hybrid_clock.cc:648] HybridClock initialized: now 1790679025185497 us; error 43 us; skew 500 ppm
Sep 29 10:50:25 dist-test-slave-vqch krb5kdc[26904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679025, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:25.188125 27702 init.cc:377] Logged in from keytab as oryx/127.20.80.193@KRBTEST.COM (short username oryx)
I20260929 10:50:25.188625 27702 webserver.cc:460] Webserver started at http://127.20.80.193:34809/ using document root <none> and password file <none>
I20260929 10:50:25.188881 27702 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:25.188964 27702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:25.190917 27702 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:50:25.191597 27807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:25.191829 27702 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:50:25.191942 27702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-0/wal
uuid: "3da20f0055b74c0292fa05c61074e4d6"
format_stamp: "Formatted at 2026-09-29 10:50:24 on dist-test-slave-vqch"
server_key: "65a9e623a43fc0b0f4b7f4f6048c2c0f"
server_key_iv: "8bf6bedac128355cdf306bb929424d88"
server_key_version: "encryptionkey@0"
I20260929 10:50:25.192466 27702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:25.214572 27702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:25.217082 27702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:25.217239 27702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:25.217469 27702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:25.217761 27702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:25.217813 27702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:25.217852 27702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:25.217882 27702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:25.223712 27702 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.193:41319
I20260929 10:50:25.223820 27921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.193:41319 every 8 connection(s)
I20260929 10:50:25.224116 27702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
I20260929 10:50:25.227036 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 27702
I20260929 10:50:25.227192 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 27147
Sep 29 10:50:25 dist-test-slave-vqch krb5kdc[26904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679025, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM
Sep 29 10:50:25 dist-test-slave-vqch krb5kdc[26904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679025, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.253@KRBTEST.COM
Sep 29 10:50:25 dist-test-slave-vqch krb5kdc[26904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679025, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.252@KRBTEST.COM
I20260929 10:50:25.235970 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.194:45001
--local_ip_for_outbound_sockets=127.20.80.194
--tserver_master_addrs=127.20.80.254:44567
--webserver_port=45883
--webserver_interface=127.20.80.194
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.194.keytab
--principal=oryx/127.20.80.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:40769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.20.80.254:44567,127.20.80.253:44859
--tserver_master_addrs=127.20.80.254:44567,127.20.80.253:44859,127.20.80.252:39713 with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:50:25.237293 26998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.225181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:53869 (local address 127.20.80.254:44567)
0929 10:50:25.225239 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:25.225242 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:25.225254 (+ 12us) server_negotiation.cc:407] Connection header received
0929 10:50:25.226254 (+ 1000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:25.226255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:25.226268 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:25.226326 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:25.227348 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.228201 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.229583 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.229944 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.229987 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.232911 (+ 2924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:25.232913 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:25.232913 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:25.232921 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:25.234731 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.235849 (+ 1118us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.235850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.235851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.236205 (+ 354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.236513 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.236514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.236514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.236646 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:25.236733 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:25.237097 (+ 364us) server_negotiation.cc:299] Negotiation successful
0929 10:50:25.237197 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 10:50:25.237514 27927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.224838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.20.80.193:53869)
0929 10:50:25.225036 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:25.225077 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:25.226142 (+ 1065us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:25.226400 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:25.226404 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:25.226770 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:25.227072 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.227079 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.228390 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.228391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.229482 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.229485 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.230067 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.230067 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.230191 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.230891 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:25.230901 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:25.232805 (+ 1904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:25.234820 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:25.234828 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:25.234830 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:25.235736 (+ 906us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:25.236294 (+ 558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:25.236295 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:25.236295 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:25.236425 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:25.236723 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:25.236731 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:25.236879 (+ 148us) client_negotiation.cc:769] Sending connection context
0929 10:50:25.237071 (+ 192us) client_negotiation.cc:240] Negotiation successful
0929 10:50:25.237237 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":43,"thread_start_us":102,"threads_started":1}
I20260929 10:50:25.238044 27923 heartbeater.cc:347] Connected to a master server at 127.20.80.254:44567
I20260929 10:50:25.238181 27923 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:25.238371 27923 heartbeater.cc:511] Master 127.20.80.254:44567 requested a full tablet report, sending...
I20260929 10:50:25.238396 27604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.225165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:42685 (local address 127.20.80.253:44859)
0929 10:50:25.225238 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:25.225240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:25.225425 (+ 185us) server_negotiation.cc:407] Connection header received
0929 10:50:25.226285 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:25.226287 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:25.226301 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:25.226340 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:25.227204 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.227969 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.229596 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.229893 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.229941 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.234130 (+ 4189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:25.234132 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:25.234132 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:25.234141 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:25.235953 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.237211 (+ 1258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.237213 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.237213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.237281 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.237514 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.237515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.237516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.237666 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:25.237718 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:25.238209 (+ 491us) server_negotiation.cc:299] Negotiation successful
0929 10:50:25.238292 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 10:50:25.238770 27929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.224838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:44859 (local address 127.20.80.193:42685)
0929 10:50:25.225338 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:25.225349 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:25.226173 (+ 824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:25.226434 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:25.226436 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:25.226800 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:25.227072 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.227079 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.228116 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.228117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.229482 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.229484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.230044 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.230044 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.230189 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.231057 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:25.231062 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:25.234011 (+ 2949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:25.236061 (+ 2050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:25.236062 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:25.236062 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:25.237120 (+ 1058us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:25.237354 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:25.237355 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:25.237355 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:25.237435 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:25.237752 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:25.237753 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:25.237796 (+ 43us) client_negotiation.cc:769] Sending connection context
0929 10:50:25.238505 (+ 709us) client_negotiation.cc:240] Negotiation successful
0929 10:50:25.238614 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"mutex_wait_us":44,"thread_start_us":141,"threads_started":1}
I20260929 10:50:25.238843 26950 ts_manager.cc:194] Re-registered known tserver with Master: 3da20f0055b74c0292fa05c61074e4d6 (127.20.80.193:41319)
I20260929 10:50:25.239256 26950 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.80.193@KRBTEST.COM'} at 127.20.80.193:53869
I20260929 10:50:25.239289 27924 heartbeater.cc:347] Connected to a master server at 127.20.80.253:44859
I20260929 10:50:25.239334 27924 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:25.239408 27924 heartbeater.cc:511] Master 127.20.80.253:44859 requested a full tablet report, sending...
I20260929 10:50:25.239758 27553 ts_manager.cc:194] Registered new tserver with Master: 3da20f0055b74c0292fa05c61074e4d6 (127.20.80.193:41319)
I20260929 10:50:25.240773 27926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.224817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:49273 (local address 127.20.80.252:39713)
0929 10:50:25.224931 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:25.224933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:25.225447 (+ 514us) server_negotiation.cc:407] Connection header received
0929 10:50:25.226286 (+ 839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:25.226287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:25.226301 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:25.226354 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:25.227198 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.228254 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.229596 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.229919 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.229958 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.235015 (+ 5057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:25.235016 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:25.235016 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:25.235023 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:25.236837 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.239960 (+ 3123us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.239961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.239962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.240027 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.240306 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.240307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.240308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.240419 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:25.240475 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:25.240646 (+ 171us) server_negotiation.cc:299] Negotiation successful
0929 10:50:25.240712 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":58,"threads_started":1}
I20260929 10:50:25.240957 27928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.224837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:39713 (local address 127.20.80.193:49273)
0929 10:50:25.225337 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:25.225349 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:25.226169 (+ 820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:25.226439 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:25.226440 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:25.226791 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:25.227072 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.227079 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.228353 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.228354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.229482 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.229484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.230046 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.230047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.230189 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.230925 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:25.230930 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:25.234927 (+ 3997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:25.237884 (+ 2957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:25.237885 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:25.237886 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:25.238097 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:25.240135 (+ 2038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:25.240136 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:25.240136 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:25.240202 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:25.240493 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:25.240494 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:25.240543 (+ 49us) client_negotiation.cc:769] Sending connection context
0929 10:50:25.240615 (+ 72us) client_negotiation.cc:240] Negotiation successful
0929 10:50:25.240741 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":11,"thread_start_us":139,"threads_started":1}
I20260929 10:50:25.241370 27922 heartbeater.cc:347] Connected to a master server at 127.20.80.252:39713
I20260929 10:50:25.241422 27922 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:25.241494 27922 heartbeater.cc:511] Master 127.20.80.252:39713 requested a full tablet report, sending...
I20260929 10:50:25.241855 27741 ts_manager.cc:194] Registered new tserver with Master: 3da20f0055b74c0292fa05c61074e4d6 (127.20.80.193:41319)
W20260929 10:50:25.318499 27930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:25.318693 27930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:25.318717 27930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:25.319802 27930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:25.319836 27930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:25.319864 27930 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.194
I20260929 10:50:25.321386 27930 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:40769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.194.keytab
--principal=oryx/127.20.80.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.194:45001
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.80.194
--webserver_port=45883
--tserver_master_addrs=127.20.80.254:44567,127.20.80.253:44859,127.20.80.252:39713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.194
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:25.321568 27930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:25.321740 27930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:25.326812 27936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:25.326800 27935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:25.326792 27938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:25.326907 27930 server_base.cc:1034] running on GCE node
I20260929 10:50:25.327194 27930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:25.327442 27930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:25.328608 27930 hybrid_clock.cc:648] HybridClock initialized: now 1790679025328592 us; error 30 us; skew 500 ppm
Sep 29 10:50:25 dist-test-slave-vqch krb5kdc[26904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679025, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:25.331146 27930 init.cc:377] Logged in from keytab as oryx/127.20.80.194@KRBTEST.COM (short username oryx)
I20260929 10:50:25.331504 27930 webserver.cc:460] Webserver started at http://127.20.80.194:45883/ using document root <none> and password file <none>
I20260929 10:50:25.331696 27930 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:25.331746 27930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:25.333045 27930 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:50:25.333552 27945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:25.333715 27930 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:25.333801 27930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-1/wal
uuid: "68952289b9a242f7a23f07cbcb8b36f2"
format_stamp: "Formatted at 2026-09-29 10:50:24 on dist-test-slave-vqch"
server_key: "4795e1c26477f56d11ba6e568da2d57d"
server_key_iv: "ebb66db63f488b19154f74457b1dda75"
server_key_version: "encryptionkey@0"
I20260929 10:50:25.334161 27930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:25.352152 27930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:25.354676 27930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:25.354848 27930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:25.355098 27930 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:25.355422 27930 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:25.355496 27930 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:25.355536 27930 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:25.355568 27930 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:25.361593 27930 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.194:45001
I20260929 10:50:25.361677 28059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.194:45001 every 8 connection(s)
I20260929 10:50:25.362028 27930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
Sep 29 10:50:25 dist-test-slave-vqch krb5kdc[26904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679025, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM
Sep 29 10:50:25 dist-test-slave-vqch krb5kdc[26904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679025, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.253@KRBTEST.COM
Sep 29 10:50:25 dist-test-slave-vqch krb5kdc[26904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679025, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.252@KRBTEST.COM
I20260929 10:50:25.371464 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 27930
I20260929 10:50:25.371891 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 27282
I20260929 10:50:25.373054 27604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.363058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:34765 (local address 127.20.80.253:44859)
0929 10:50:25.363107 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:25.363110 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:25.363149 (+ 39us) server_negotiation.cc:407] Connection header received
0929 10:50:25.363669 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:25.363670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:25.363683 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:25.363733 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:25.364633 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.365425 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.366544 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.366752 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.366837 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.369801 (+ 2964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:25.369802 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:25.369803 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:25.369810 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:25.371416 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.371891 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.371893 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.371893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.371956 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.372381 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.372382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.372383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.372546 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:25.372590 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:25.372844 (+ 254us) server_negotiation.cc:299] Negotiation successful
0929 10:50:25.372925 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260929 10:50:25.373106 26998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.362889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:40533 (local address 127.20.80.254:44567)
0929 10:50:25.362940 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:25.362942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:25.364121 (+ 1179us) server_negotiation.cc:407] Connection header received
0929 10:50:25.364210 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:25.364211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:25.364224 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:25.364287 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:25.364637 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.365425 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.366523 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.366735 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.366787 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.369337 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:25.369338 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:25.369338 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:25.369345 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:25.371174 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.371996 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.371997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.371997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.372055 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.372244 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.372244 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.372245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.372409 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:25.372459 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:25.372850 (+ 391us) server_negotiation.cc:299] Negotiation successful
0929 10:50:25.372942 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 10:50:25.373286 28064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.362791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:44859 (local address 127.20.80.194:34765)
0929 10:50:25.363028 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:25.363073 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:25.363562 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:25.363817 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:25.363823 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:25.364307 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:25.364523 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.364528 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.365523 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.365524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.366438 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.366440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.366857 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.366858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.366952 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.367593 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:25.367603 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:25.369720 (+ 2117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:25.371495 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:25.371502 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:25.371505 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:25.371777 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:25.372035 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:25.372036 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:25.372037 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:25.372163 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:25.372641 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:25.372641 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:25.372689 (+ 48us) client_negotiation.cc:769] Sending connection context
0929 10:50:25.372816 (+ 127us) client_negotiation.cc:240] Negotiation successful
0929 10:50:25.372983 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":27,"thread_start_us":80,"threads_started":1}
I20260929 10:50:25.373507 28065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.362791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.20.80.194:40533)
0929 10:50:25.364058 (+ 1267us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:25.364068 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:25.364129 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:25.364313 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:25.364314 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:25.364464 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:25.364522 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.364528 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.365523 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.365524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.366438 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.366440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.366817 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.366818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.366946 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.367682 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:25.367687 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:25.369249 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:25.371612 (+ 2363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:25.371613 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:25.371614 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:25.371904 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:25.372124 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:25.372125 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:25.372125 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:25.372182 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:25.372506 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:25.372514 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:25.372689 (+ 175us) client_negotiation.cc:769] Sending connection context
0929 10:50:25.373225 (+ 536us) client_negotiation.cc:240] Negotiation successful
0929 10:50:25.373340 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1029,"mutex_wait_us":208,"thread_start_us":85,"threads_started":1}
I20260929 10:50:25.373875 28061 heartbeater.cc:347] Connected to a master server at 127.20.80.254:44567
I20260929 10:50:25.374092 28062 heartbeater.cc:347] Connected to a master server at 127.20.80.253:44859
I20260929 10:50:25.374095 28061 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:25.374182 28062 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:25.374344 28062 heartbeater.cc:511] Master 127.20.80.253:44859 requested a full tablet report, sending...
I20260929 10:50:25.374358 28061 heartbeater.cc:511] Master 127.20.80.254:44567 requested a full tablet report, sending...
I20260929 10:50:25.374850 27553 ts_manager.cc:194] Registered new tserver with Master: 68952289b9a242f7a23f07cbcb8b36f2 (127.20.80.194:45001)
I20260929 10:50:25.375007 26950 ts_manager.cc:194] Re-registered known tserver with Master: 68952289b9a242f7a23f07cbcb8b36f2 (127.20.80.194:45001)
I20260929 10:50:25.375375 26950 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.80.194@KRBTEST.COM'} at 127.20.80.194:40533
I20260929 10:50:25.378794 28066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.362792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:39713 (local address 127.20.80.194:60521)
0929 10:50:25.363090 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:25.363102 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:25.363576 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:25.363829 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:25.363829 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:25.364307 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:25.364522 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.364527 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.365322 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.365323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.366141 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.366142 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.366859 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.366859 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.366952 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.367619 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:25.367623 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:25.370580 (+ 2957us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:25.377529 (+ 6949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:25.377530 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:25.377530 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:25.377777 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:25.377993 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:25.377994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:25.377995 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:25.378112 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:25.378378 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:25.378379 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:25.378430 (+ 51us) client_negotiation.cc:769] Sending connection context
0929 10:50:25.378507 (+ 77us) client_negotiation.cc:240] Negotiation successful
0929 10:50:25.378633 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":36,"thread_start_us":69,"threads_started":1}
I20260929 10:50:25.378852 27926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.362899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:60521 (local address 127.20.80.252:39713)
0929 10:50:25.362944 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:25.362945 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:25.363162 (+ 217us) server_negotiation.cc:407] Connection header received
0929 10:50:25.363663 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:25.363664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:25.363679 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:25.363739 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:25.364645 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.365218 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.366236 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.366453 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.366531 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.370822 (+ 4291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:25.370824 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:25.370824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:25.370831 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:25.372527 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.377868 (+ 5341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.377869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.377870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.377926 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.378175 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.378176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.378176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.378295 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:25.378518 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:25.378620 (+ 102us) server_negotiation.cc:299] Negotiation successful
0929 10:50:25.378732 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 10:50:25.380249 28060 heartbeater.cc:347] Connected to a master server at 127.20.80.252:39713
I20260929 10:50:25.380304 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.195:35313
--local_ip_for_outbound_sockets=127.20.80.195
--tserver_master_addrs=127.20.80.254:44567
--webserver_port=46135
--webserver_interface=127.20.80.195
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.195.keytab
--principal=oryx/127.20.80.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:40769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.20.80.254:44567,127.20.80.253:44859
--tserver_master_addrs=127.20.80.254:44567,127.20.80.253:44859,127.20.80.252:39713 with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:50:25.380348 28060 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:25.380432 28060 heartbeater.cc:511] Master 127.20.80.252:39713 requested a full tablet report, sending...
I20260929 10:50:25.380779 27741 ts_manager.cc:194] Registered new tserver with Master: 68952289b9a242f7a23f07cbcb8b36f2 (127.20.80.194:45001)
W20260929 10:50:25.467936 28067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:25.468164 28067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:25.468191 28067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:25.469343 28067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:25.469390 28067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:25.469416 28067 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.195
I20260929 10:50:25.470870 28067 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:40769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.195.keytab
--principal=oryx/127.20.80.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.195:35313
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.80.195
--webserver_port=46135
--tserver_master_addrs=127.20.80.254:44567,127.20.80.253:44859,127.20.80.252:39713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.195
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:25.471083 28067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:25.471272 28067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 10:50:25.476722 28067 server_base.cc:1034] running on GCE node
W20260929 10:50:25.476719 28073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:25.476742 28072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:25.476719 28075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:25.477244 28067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:25.477497 28067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:25.478684 28067 hybrid_clock.cc:648] HybridClock initialized: now 1790679025478680 us; error 56 us; skew 500 ppm
Sep 29 10:50:25 dist-test-slave-vqch krb5kdc[26904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679025, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:25.481262 28067 init.cc:377] Logged in from keytab as oryx/127.20.80.195@KRBTEST.COM (short username oryx)
I20260929 10:50:25.481720 28067 webserver.cc:460] Webserver started at http://127.20.80.195:46135/ using document root <none> and password file <none>
I20260929 10:50:25.481925 28067 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:25.481977 28067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:25.483345 28067 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:50:25.484012 28082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:25.484221 28067 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:50:25.484292 28067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-2/wal
uuid: "658a8336746b42189751b2817f40a623"
format_stamp: "Formatted at 2026-09-29 10:50:24 on dist-test-slave-vqch"
server_key: "56f89ef33ad2da08dcde4edea74964d9"
server_key_iv: "125e6484b2e51df6b4bda59af997d795"
server_key_version: "encryptionkey@0"
I20260929 10:50:25.484644 28067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:25.499125 28067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:25.501497 28067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:25.501645 28067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:25.501896 28067 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:25.502203 28067 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:25.502236 28067 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:25.502259 28067 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:25.502305 28067 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:25.507973 28067 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.195:35313
I20260929 10:50:25.508028 28195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.195:35313 every 8 connection(s)
I20260929 10:50:25.508385 28067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
Sep 29 10:50:25 dist-test-slave-vqch krb5kdc[26904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679025, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.252@KRBTEST.COM
Sep 29 10:50:25 dist-test-slave-vqch krb5kdc[26904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679025, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM
I20260929 10:50:25.515820 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 28067
Sep 29 10:50:25 dist-test-slave-vqch krb5kdc[26904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679025, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.253@KRBTEST.COM
I20260929 10:50:25.519013 27926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.509084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:56271 (local address 127.20.80.252:39713)
0929 10:50:25.509565 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:25.509567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:25.509643 (+ 76us) server_negotiation.cc:407] Connection header received
0929 10:50:25.510596 (+ 953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:25.510597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:25.510609 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:25.510661 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:25.511421 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.511915 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.512938 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.513183 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.513271 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.515577 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:25.515579 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:25.515579 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:25.515588 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:25.517335 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.517846 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.517847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.517848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.517912 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.518223 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.518224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.518224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.518367 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:25.518412 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:25.518804 (+ 392us) server_negotiation.cc:299] Negotiation successful
0929 10:50:25.518899 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449}
I20260929 10:50:25.519732 28202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.509094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:39713 (local address 127.20.80.195:56271)
0929 10:50:25.509579 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:25.509590 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:25.510496 (+ 906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:25.510742 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:25.510746 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:25.511106 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:25.511302 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.511306 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.512018 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.512019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.512832 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.512834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.513278 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.513279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.513407 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.513898 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:25.513905 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:25.515484 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:25.517453 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:25.517460 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:25.517462 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:25.517760 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:25.517998 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:25.517999 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:25.517999 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:25.518138 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:25.518452 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:25.518478 (+ 26us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:25.518635 (+ 157us) client_negotiation.cc:769] Sending connection context
0929 10:50:25.519262 (+ 627us) client_negotiation.cc:240] Negotiation successful
0929 10:50:25.519471 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"mutex_wait_us":36,"thread_start_us":83,"threads_started":1}
I20260929 10:50:25.520430 27604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.509180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:58491 (local address 127.20.80.253:44859)
0929 10:50:25.509237 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:25.509239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:25.509454 (+ 215us) server_negotiation.cc:407] Connection header received
0929 10:50:25.510617 (+ 1163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:25.510618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:25.510631 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:25.510664 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:25.511409 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.512168 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.513310 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.513603 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.513644 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.517416 (+ 3772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:25.517417 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:25.517418 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:25.517425 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:25.519235 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.519646 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.519647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.519647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.519702 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.519904 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.519905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.519905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.520035 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:25.520116 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:25.520266 (+ 150us) server_negotiation.cc:299] Negotiation successful
0929 10:50:25.520346 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 10:50:25.520445 28201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.509094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:44859 (local address 127.20.80.195:58491)
0929 10:50:25.509311 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:25.509353 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:25.510494 (+ 1141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:25.510743 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:25.510747 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:25.511112 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:25.511302 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.511309 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.512257 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.512258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.513204 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.513205 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.513722 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.513723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.513790 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.514388 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:25.514393 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:25.517149 (+ 2756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:25.519333 (+ 2184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:25.519333 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:25.519334 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:25.519566 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:25.519769 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:25.519770 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:25.519770 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:25.519830 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:25.520109 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:25.520110 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:25.520151 (+ 41us) client_negotiation.cc:769] Sending connection context
0929 10:50:25.520230 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 10:50:25.520329 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":75,"thread_start_us":90,"threads_started":1}
I20260929 10:50:25.520879 28198 heartbeater.cc:347] Connected to a master server at 127.20.80.253:44859
I20260929 10:50:25.520995 28198 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:25.521106 28196 heartbeater.cc:347] Connected to a master server at 127.20.80.252:39713
I20260929 10:50:25.521155 28196 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:25.521219 28196 heartbeater.cc:511] Master 127.20.80.252:39713 requested a full tablet report, sending...
I20260929 10:50:25.521191 28198 heartbeater.cc:511] Master 127.20.80.253:44859 requested a full tablet report, sending...
I20260929 10:50:25.521292 26998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.509527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:33925 (local address 127.20.80.254:44567)
0929 10:50:25.509582 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:25.509584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:25.509675 (+ 91us) server_negotiation.cc:407] Connection header received
0929 10:50:25.510618 (+ 943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:25.510619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:25.510631 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:25.510664 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:25.511410 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.512162 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.513314 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.513607 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.513644 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.517999 (+ 4355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:25.518001 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:25.518002 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:25.518010 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:25.519992 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.520361 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.520362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.520362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.520425 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.520665 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.520666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.520667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.520804 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:25.520848 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:25.521116 (+ 268us) server_negotiation.cc:299] Negotiation successful
0929 10:50:25.521201 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260929 10:50:25.521314 28200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.509445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.20.80.195:33925)
0929 10:50:25.509615 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:25.509624 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:25.510499 (+ 875us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:25.510743 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:25.510747 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:25.511120 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:25.511302 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.511314 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.512257 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.512258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.513201 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.513204 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.513723 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.513723 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.513790 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.514295 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:25.514299 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:25.517916 (+ 3617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:25.520060 (+ 2144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:25.520060 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:25.520061 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:25.520274 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:25.520533 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:25.520533 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:25.520534 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:25.520597 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:25.520919 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:25.520919 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:25.520961 (+ 42us) client_negotiation.cc:769] Sending connection context
0929 10:50:25.521079 (+ 118us) client_negotiation.cc:240] Negotiation successful
0929 10:50:25.521181 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":61,"thread_start_us":64,"threads_started":1}
I20260929 10:50:25.521612 28197 heartbeater.cc:347] Connected to a master server at 127.20.80.254:44567
I20260929 10:50:25.521651 28197 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:25.521703 28197 heartbeater.cc:511] Master 127.20.80.254:44567 requested a full tablet report, sending...
I20260929 10:50:25.521672 27741 ts_manager.cc:194] Registered new tserver with Master: 658a8336746b42189751b2817f40a623 (127.20.80.195:35313)
I20260929 10:50:25.521746 27553 ts_manager.cc:194] Registered new tserver with Master: 658a8336746b42189751b2817f40a623 (127.20.80.195:35313)
I20260929 10:50:25.521899 26950 ts_manager.cc:194] Re-registered known tserver with Master: 658a8336746b42189751b2817f40a623 (127.20.80.195:35313)
I20260929 10:50:25.522347 26950 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.80.195@KRBTEST.COM'} at 127.20.80.195:33925
I20260929 10:50:25.527325 28208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.517257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52476 (local address 127.20.80.254:44567)
0929 10:50:25.518047 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:25.518050 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:25.518057 (+ 7us) server_negotiation.cc:407] Connection header received
0929 10:50:25.518075 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:25.518075 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:25.518087 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:25.518151 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:25.519046 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.519868 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.522444 (+ 2576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.522765 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.523187 (+ 422us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.524173 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:25.524174 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:25.524175 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:25.524181 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:25.525897 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.526328 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.526329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.526330 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.526388 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.526607 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.526608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.526608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.526741 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:25.526862 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:25.527111 (+ 249us) server_negotiation.cc:299] Negotiation successful
0929 10:50:25.527196 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":760,"thread_start_us":85,"threads_started":1}
I20260929 10:50:25.530260 27604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.517328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58110 (local address 127.20.80.253:44859)
0929 10:50:25.520505 (+ 3177us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:25.520507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:25.522719 (+ 2212us) server_negotiation.cc:407] Connection header received
0929 10:50:25.522890 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:25.522891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:25.522900 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:25.522928 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:25.523642 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.524346 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.525333 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.525531 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.525615 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.526985 (+ 1370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:25.526987 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:25.526987 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:25.526994 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:25.529100 (+ 2106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.529434 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.529435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.529435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.529488 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.529709 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.529710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.529710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.529832 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:25.529911 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:25.530105 (+ 194us) server_negotiation.cc:299] Negotiation successful
0929 10:50:25.530175 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3161,"thread_start_us":58,"threads_started":1}
I20260929 10:50:25.530445 27926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.517457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35844 (local address 127.20.80.252:39713)
0929 10:50:25.519087 (+ 1630us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:25.519089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:25.522526 (+ 3437us) server_negotiation.cc:407] Connection header received
0929 10:50:25.522837 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:25.522838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:25.522850 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:25.522901 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:25.524243 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.525017 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.526111 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.526457 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.526540 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.527777 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:25.527778 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:25.527779 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:25.527785 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:25.529296 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.529681 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.529682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.529682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.529733 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.529943 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.529944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.529944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.530062 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:25.530159 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:25.530274 (+ 115us) server_negotiation.cc:299] Negotiation successful
0929 10:50:25.530347 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1614,"thread_start_us":52,"threads_started":1}
I20260929 10:50:25.529034 26950 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52476:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:50:25.531453 26950 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:50:25.539264 28219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.535146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:45001 (local address 127.0.0.1:53478)
0929 10:50:25.535331 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:25.535339 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:25.535388 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:25.535934 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:25.535935 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:25.535937 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:25.535978 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.535980 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.537176 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.537176 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.538102 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.538104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.539013 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.539014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.539070 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.539074 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:50:25.539108 (+ 34us) client_negotiation.cc:240] Negotiation successful
0929 10:50:25.539155 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":27,"thread_start_us":77,"threads_started":1}
I20260929 10:50:25.539286 28218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.535009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41932 (local address 127.20.80.193:41319)
0929 10:50:25.535516 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:25.535521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:25.535530 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:50:25.535571 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:25.535573 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:25.535589 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:25.535669 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:25.535790 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.536893 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.538261 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.538939 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.539015 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.539096 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:25.539169 (+ 73us) server_negotiation.cc:299] Negotiation successful
0929 10:50:25.539213 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":87,"threads_started":1}
I20260929 10:50:25.539337 28217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.534960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:35313 (local address 127.0.0.1:49190)
0929 10:50:25.535084 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:25.535094 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:25.535230 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:25.536209 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:25.536210 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:25.536211 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:25.536251 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.536252 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.537272 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.537273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.538380 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.538382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.539102 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.539104 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.539165 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.539168 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:50:25.539196 (+ 28us) client_negotiation.cc:240] Negotiation successful
0929 10:50:25.539228 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":80,"threads_started":1}
I20260929 10:50:25.539453 27512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.534898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:41319 (local address 127.0.0.1:41932)
0929 10:50:25.534929 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:25.534939 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:25.534976 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:25.535683 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:25.535683 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:25.535685 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:25.535726 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.535728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.536973 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.536994 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.538193 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.538196 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.539025 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.539026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.539092 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.539096 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:50:25.539154 (+ 58us) client_negotiation.cc:240] Negotiation successful
0929 10:50:25.539196 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16,"mutex_wait_us":31}
I20260929 10:50:25.539485 28220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.535233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53478 (local address 127.20.80.194:45001)
0929 10:50:25.535791 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:25.535794 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:25.535802 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:50:25.535832 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:25.535833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:25.535845 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:25.535935 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:25.536046 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.537098 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.538195 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.538905 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.539243 (+ 338us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.539337 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:25.539365 (+ 28us) server_negotiation.cc:299] Negotiation successful
0929 10:50:25.539411 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":88,"threads_started":1}
I20260929 10:50:25.539746 28221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.535359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49190 (local address 127.20.80.195:35313)
0929 10:50:25.536054 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:25.536057 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:25.536066 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:50:25.536098 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:25.536099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:25.536121 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:25.536199 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:25.536301 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.537178 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.538470 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.539011 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.539520 (+ 509us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.539617 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:25.539642 (+ 25us) server_negotiation.cc:299] Negotiation successful
0929 10:50:25.539678 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":82,"threads_started":1}
I20260929 10:50:25.540181 27855 tablet_service.cc:1467] Processing CreateTablet for tablet a66c8a22365e4e979d8ac59a7a63c90d (DEFAULT_TABLE table=test-table [id=d399fcf1cdbc476fb050a6cee503466a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:25.540592 27855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a66c8a22365e4e979d8ac59a7a63c90d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:25.540599 28130 tablet_service.cc:1467] Processing CreateTablet for tablet a66c8a22365e4e979d8ac59a7a63c90d (DEFAULT_TABLE table=test-table [id=d399fcf1cdbc476fb050a6cee503466a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:25.540598 27993 tablet_service.cc:1467] Processing CreateTablet for tablet a66c8a22365e4e979d8ac59a7a63c90d (DEFAULT_TABLE table=test-table [id=d399fcf1cdbc476fb050a6cee503466a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:25.540917 28130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a66c8a22365e4e979d8ac59a7a63c90d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:25.540926 27993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a66c8a22365e4e979d8ac59a7a63c90d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:25.543605 28222 tablet_bootstrap.cc:492] T a66c8a22365e4e979d8ac59a7a63c90d P 3da20f0055b74c0292fa05c61074e4d6: Bootstrap starting.
I20260929 10:50:25.544282 28223 tablet_bootstrap.cc:492] T a66c8a22365e4e979d8ac59a7a63c90d P 658a8336746b42189751b2817f40a623: Bootstrap starting.
I20260929 10:50:25.544404 28222 tablet_bootstrap.cc:654] T a66c8a22365e4e979d8ac59a7a63c90d P 3da20f0055b74c0292fa05c61074e4d6: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:25.544745 28222 log.cc:824] T a66c8a22365e4e979d8ac59a7a63c90d P 3da20f0055b74c0292fa05c61074e4d6: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:25.544941 28223 tablet_bootstrap.cc:654] T a66c8a22365e4e979d8ac59a7a63c90d P 658a8336746b42189751b2817f40a623: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:25.545123 28224 tablet_bootstrap.cc:492] T a66c8a22365e4e979d8ac59a7a63c90d P 68952289b9a242f7a23f07cbcb8b36f2: Bootstrap starting.
I20260929 10:50:25.545292 28223 log.cc:824] T a66c8a22365e4e979d8ac59a7a63c90d P 658a8336746b42189751b2817f40a623: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:25.545774 28224 tablet_bootstrap.cc:654] T a66c8a22365e4e979d8ac59a7a63c90d P 68952289b9a242f7a23f07cbcb8b36f2: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:25.546049 28223 tablet_bootstrap.cc:492] T a66c8a22365e4e979d8ac59a7a63c90d P 658a8336746b42189751b2817f40a623: No bootstrap required, opened a new log
I20260929 10:50:25.546073 28224 log.cc:824] T a66c8a22365e4e979d8ac59a7a63c90d P 68952289b9a242f7a23f07cbcb8b36f2: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:25.546120 28223 ts_tablet_manager.cc:1397] T a66c8a22365e4e979d8ac59a7a63c90d P 658a8336746b42189751b2817f40a623: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260929 10:50:25.546612 28222 tablet_bootstrap.cc:492] T a66c8a22365e4e979d8ac59a7a63c90d P 3da20f0055b74c0292fa05c61074e4d6: No bootstrap required, opened a new log
I20260929 10:50:25.546684 28222 ts_tablet_manager.cc:1397] T a66c8a22365e4e979d8ac59a7a63c90d P 3da20f0055b74c0292fa05c61074e4d6: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260929 10:50:25.546867 28224 tablet_bootstrap.cc:492] T a66c8a22365e4e979d8ac59a7a63c90d P 68952289b9a242f7a23f07cbcb8b36f2: No bootstrap required, opened a new log
I20260929 10:50:25.546933 28224 ts_tablet_manager.cc:1397] T a66c8a22365e4e979d8ac59a7a63c90d P 68952289b9a242f7a23f07cbcb8b36f2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:50:25.548663 28223 raft_consensus.cc:348] T a66c8a22365e4e979d8ac59a7a63c90d P 658a8336746b42189751b2817f40a623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3da20f0055b74c0292fa05c61074e4d6" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 41319 } } peers { permanent_uuid: "658a8336746b42189751b2817f40a623" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35313 } } peers { permanent_uuid: "68952289b9a242f7a23f07cbcb8b36f2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 45001 } }
I20260929 10:50:25.548840 28223 raft_consensus.cc:374] T a66c8a22365e4e979d8ac59a7a63c90d P 658a8336746b42189751b2817f40a623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:25.548741 28224 raft_consensus.cc:348] T a66c8a22365e4e979d8ac59a7a63c90d P 68952289b9a242f7a23f07cbcb8b36f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3da20f0055b74c0292fa05c61074e4d6" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 41319 } } peers { permanent_uuid: "658a8336746b42189751b2817f40a623" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35313 } } peers { permanent_uuid: "68952289b9a242f7a23f07cbcb8b36f2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 45001 } }
I20260929 10:50:25.548883 28223 raft_consensus.cc:729] T a66c8a22365e4e979d8ac59a7a63c90d P 658a8336746b42189751b2817f40a623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 658a8336746b42189751b2817f40a623, State: Initialized, Role: FOLLOWER
I20260929 10:50:25.548887 28224 raft_consensus.cc:374] T a66c8a22365e4e979d8ac59a7a63c90d P 68952289b9a242f7a23f07cbcb8b36f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:25.548923 28224 raft_consensus.cc:729] T a66c8a22365e4e979d8ac59a7a63c90d P 68952289b9a242f7a23f07cbcb8b36f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68952289b9a242f7a23f07cbcb8b36f2, State: Initialized, Role: FOLLOWER
I20260929 10:50:25.549050 28223 consensus_queue.cc:260] T a66c8a22365e4e979d8ac59a7a63c90d P 658a8336746b42189751b2817f40a623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3da20f0055b74c0292fa05c61074e4d6" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 41319 } } peers { permanent_uuid: "658a8336746b42189751b2817f40a623" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35313 } } peers { permanent_uuid: "68952289b9a242f7a23f07cbcb8b36f2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 45001 } }
I20260929 10:50:25.549059 28224 consensus_queue.cc:260] T a66c8a22365e4e979d8ac59a7a63c90d P 68952289b9a242f7a23f07cbcb8b36f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3da20f0055b74c0292fa05c61074e4d6" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 41319 } } peers { permanent_uuid: "658a8336746b42189751b2817f40a623" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35313 } } peers { permanent_uuid: "68952289b9a242f7a23f07cbcb8b36f2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 45001 } }
I20260929 10:50:25.549167 28222 raft_consensus.cc:348] T a66c8a22365e4e979d8ac59a7a63c90d P 3da20f0055b74c0292fa05c61074e4d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3da20f0055b74c0292fa05c61074e4d6" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 41319 } } peers { permanent_uuid: "658a8336746b42189751b2817f40a623" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35313 } } peers { permanent_uuid: "68952289b9a242f7a23f07cbcb8b36f2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 45001 } }
I20260929 10:50:25.549327 28223 ts_tablet_manager.cc:1428] T a66c8a22365e4e979d8ac59a7a63c90d P 658a8336746b42189751b2817f40a623: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260929 10:50:25.549327 28222 raft_consensus.cc:374] T a66c8a22365e4e979d8ac59a7a63c90d P 3da20f0055b74c0292fa05c61074e4d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:25.549328 28224 ts_tablet_manager.cc:1428] T a66c8a22365e4e979d8ac59a7a63c90d P 68952289b9a242f7a23f07cbcb8b36f2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 10:50:25.549366 28222 raft_consensus.cc:729] T a66c8a22365e4e979d8ac59a7a63c90d P 3da20f0055b74c0292fa05c61074e4d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da20f0055b74c0292fa05c61074e4d6, State: Initialized, Role: FOLLOWER
I20260929 10:50:25.549494 28222 consensus_queue.cc:260] T a66c8a22365e4e979d8ac59a7a63c90d P 3da20f0055b74c0292fa05c61074e4d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3da20f0055b74c0292fa05c61074e4d6" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 41319 } } peers { permanent_uuid: "658a8336746b42189751b2817f40a623" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35313 } } peers { permanent_uuid: "68952289b9a242f7a23f07cbcb8b36f2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 45001 } }
I20260929 10:50:25.549649 28197 heartbeater.cc:503] Master 127.20.80.254:44567 was elected leader, sending a full tablet report...
I20260929 10:50:25.550226 28061 heartbeater.cc:503] Master 127.20.80.254:44567 was elected leader, sending a full tablet report...
I20260929 10:50:25.550397 28222 ts_tablet_manager.cc:1428] T a66c8a22365e4e979d8ac59a7a63c90d P 3da20f0055b74c0292fa05c61074e4d6: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260929 10:50:25.550956 27923 heartbeater.cc:503] Master 127.20.80.254:44567 was elected leader, sending a full tablet report...
I20260929 10:50:25.580935 27512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.578350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:39713 (local address 127.0.0.1:35846)
0929 10:50:25.578403 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:25.578414 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:25.578460 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:25.578650 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:25.578651 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:25.578653 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:25.578708 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.578710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.579392 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.579392 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.580086 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.580088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.580691 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.580692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.580754 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.580757 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:50:25.580820 (+ 63us) client_negotiation.cc:240] Negotiation successful
0929 10:50:25.580866 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260929 10:50:25.580935 27926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.578423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35846 (local address 127.20.80.252:39713)
0929 10:50:25.578488 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:25.578489 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:25.578512 (+ 23us) server_negotiation.cc:407] Connection header received
0929 10:50:25.578536 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:25.578537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:25.578557 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:25.578608 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:25.578784 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.579305 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.580156 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.580610 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.580690 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.580776 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:25.580825 (+ 49us) server_negotiation.cc:299] Negotiation successful
0929 10:50:25.580857 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260929 10:50:25.581292 27752 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260929 10:50:25.582222 27781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39326fe5440f4d74abd39e9b375c9eaf. Latest consensus state: current_term: 1 leader_uuid: "39326fe5440f4d74abd39e9b375c9eaf" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: false } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: NON_VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: true } } }
I20260929 10:50:25.582331 27781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [sys.catalog]: This master's current role is: LEARNER
W20260929 10:50:25.614008 28063 tablet.cc:2369] T a66c8a22365e4e979d8ac59a7a63c90d P 68952289b9a242f7a23f07cbcb8b36f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:50:25.725152 27925 tablet.cc:2369] T a66c8a22365e4e979d8ac59a7a63c90d P 3da20f0055b74c0292fa05c61074e4d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:50:25.736472 28229 raft_consensus.cc:482] T a66c8a22365e4e979d8ac59a7a63c90d P 658a8336746b42189751b2817f40a623 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:50:25.736599 28229 raft_consensus.cc:504] T a66c8a22365e4e979d8ac59a7a63c90d P 658a8336746b42189751b2817f40a623 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3da20f0055b74c0292fa05c61074e4d6" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 41319 } } peers { permanent_uuid: "658a8336746b42189751b2817f40a623" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35313 } } peers { permanent_uuid: "68952289b9a242f7a23f07cbcb8b36f2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 45001 } }
I20260929 10:50:25.737067 28229 leader_election.cc:288] T a66c8a22365e4e979d8ac59a7a63c90d P 658a8336746b42189751b2817f40a623 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da20f0055b74c0292fa05c61074e4d6 (127.20.80.193:41319), 68952289b9a242f7a23f07cbcb8b36f2 (127.20.80.194:45001)
I20260929 10:50:25.740109 28218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.737339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:54281 (local address 127.20.80.193:41319)
0929 10:50:25.737383 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:25.737385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:25.737395 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:50:25.737422 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:25.737423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:25.737434 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:25.737581 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:25.737747 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.738229 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.739321 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.739808 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.739860 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.739869 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:25.740029 (+ 160us) server_negotiation.cc:299] Negotiation successful
0929 10:50:25.740060 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 10:50:25.740131 28200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.737251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:41319 (local address 127.20.80.195:54281)
0929 10:50:25.737289 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:25.737303 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:25.737352 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:25.737586 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:25.737587 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:25.737589 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:25.737664 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.737666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.738316 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.738317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.739237 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.739239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.739886 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.739887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.739957 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.739961 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:50:25.740019 (+ 58us) client_negotiation.cc:240] Negotiation successful
0929 10:50:25.740055 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260929 10:50:25.740487 27875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a66c8a22365e4e979d8ac59a7a63c90d" candidate_uuid: "658a8336746b42189751b2817f40a623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da20f0055b74c0292fa05c61074e4d6" is_pre_election: true
I20260929 10:50:25.740576 28201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.737273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:45001 (local address 127.20.80.195:34765)
0929 10:50:25.737329 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:25.737340 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:25.737384 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:25.737829 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:25.737832 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:25.737834 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:25.737923 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.737925 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.738759 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.738759 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.739565 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:25.739567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.740315 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.740316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:25.740399 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.740402 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:50:25.740450 (+ 48us) client_negotiation.cc:240] Negotiation successful
0929 10:50:25.740492 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260929 10:50:25.740605 28220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.737356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:34765 (local address 127.20.80.194:45001)
0929 10:50:25.737399 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:25.737401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:25.737409 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:50:25.737440 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:25.737441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:25.737449 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:25.737699 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:25.738002 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.738676 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.739662 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.740119 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.740221 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.740231 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:25.740496 (+ 265us) server_negotiation.cc:299] Negotiation successful
0929 10:50:25.740542 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 10:50:25.740643 27875 raft_consensus.cc:2454] T a66c8a22365e4e979d8ac59a7a63c90d P 3da20f0055b74c0292fa05c61074e4d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 658a8336746b42189751b2817f40a623 in term 0.
I20260929 10:50:25.740880 28083 leader_election.cc:302] T a66c8a22365e4e979d8ac59a7a63c90d P 658a8336746b42189751b2817f40a623 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da20f0055b74c0292fa05c61074e4d6, 658a8336746b42189751b2817f40a623; no voters:
I20260929 10:50:25.740908 28013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a66c8a22365e4e979d8ac59a7a63c90d" candidate_uuid: "658a8336746b42189751b2817f40a623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68952289b9a242f7a23f07cbcb8b36f2" is_pre_election: true
I20260929 10:50:25.741088 28013 raft_consensus.cc:2454] T a66c8a22365e4e979d8ac59a7a63c90d P 68952289b9a242f7a23f07cbcb8b36f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 658a8336746b42189751b2817f40a623 in term 0.
I20260929 10:50:25.741094 28229 raft_consensus.cc:2781] T a66c8a22365e4e979d8ac59a7a63c90d P 658a8336746b42189751b2817f40a623 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:50:25.741161 28229 raft_consensus.cc:482] T a66c8a22365e4e979d8ac59a7a63c90d P 658a8336746b42189751b2817f40a623 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:50:25.741186 28229 raft_consensus.cc:3037] T a66c8a22365e4e979d8ac59a7a63c90d P 658a8336746b42189751b2817f40a623 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:25.741899 28229 raft_consensus.cc:504] T a66c8a22365e4e979d8ac59a7a63c90d P 658a8336746b42189751b2817f40a623 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3da20f0055b74c0292fa05c61074e4d6" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 41319 } } peers { permanent_uuid: "658a8336746b42189751b2817f40a623" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35313 } } peers { permanent_uuid: "68952289b9a242f7a23f07cbcb8b36f2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 45001 } }
I20260929 10:50:25.742031 28229 leader_election.cc:288] T a66c8a22365e4e979d8ac59a7a63c90d P 658a8336746b42189751b2817f40a623 [CANDIDATE]: Term 1 election: Requested vote from peers 3da20f0055b74c0292fa05c61074e4d6 (127.20.80.193:41319), 68952289b9a242f7a23f07cbcb8b36f2 (127.20.80.194:45001)
I20260929 10:50:25.742193 28013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a66c8a22365e4e979d8ac59a7a63c90d" candidate_uuid: "658a8336746b42189751b2817f40a623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68952289b9a242f7a23f07cbcb8b36f2"
I20260929 10:50:25.742185 27875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a66c8a22365e4e979d8ac59a7a63c90d" candidate_uuid: "658a8336746b42189751b2817f40a623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da20f0055b74c0292fa05c61074e4d6"
I20260929 10:50:25.742282 28013 raft_consensus.cc:3037] T a66c8a22365e4e979d8ac59a7a63c90d P 68952289b9a242f7a23f07cbcb8b36f2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:25.742280 27875 raft_consensus.cc:3037] T a66c8a22365e4e979d8ac59a7a63c90d P 3da20f0055b74c0292fa05c61074e4d6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:25.742978 27875 raft_consensus.cc:2454] T a66c8a22365e4e979d8ac59a7a63c90d P 3da20f0055b74c0292fa05c61074e4d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658a8336746b42189751b2817f40a623 in term 1.
I20260929 10:50:25.743040 28013 raft_consensus.cc:2454] T a66c8a22365e4e979d8ac59a7a63c90d P 68952289b9a242f7a23f07cbcb8b36f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658a8336746b42189751b2817f40a623 in term 1.
I20260929 10:50:25.743130 28083 leader_election.cc:302] T a66c8a22365e4e979d8ac59a7a63c90d P 658a8336746b42189751b2817f40a623 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da20f0055b74c0292fa05c61074e4d6, 658a8336746b42189751b2817f40a623; no voters:
I20260929 10:50:25.743228 28229 raft_consensus.cc:2781] T a66c8a22365e4e979d8ac59a7a63c90d P 658a8336746b42189751b2817f40a623 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:25.743366 28229 raft_consensus.cc:686] T a66c8a22365e4e979d8ac59a7a63c90d P 658a8336746b42189751b2817f40a623 [term 1 LEADER]: Becoming Leader. State: Replica: 658a8336746b42189751b2817f40a623, State: Running, Role: LEADER
I20260929 10:50:25.743471 28229 consensus_queue.cc:237] T a66c8a22365e4e979d8ac59a7a63c90d P 658a8336746b42189751b2817f40a623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3da20f0055b74c0292fa05c61074e4d6" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 41319 } } peers { permanent_uuid: "658a8336746b42189751b2817f40a623" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35313 } } peers { permanent_uuid: "68952289b9a242f7a23f07cbcb8b36f2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 45001 } }
I20260929 10:50:25.744258 26949 catalog_manager.cc:5495] T a66c8a22365e4e979d8ac59a7a63c90d P 658a8336746b42189751b2817f40a623 reported cstate change: term changed from 0 to 1, leader changed from <none> to 658a8336746b42189751b2817f40a623 (127.20.80.195). New cstate: current_term: 1 leader_uuid: "658a8336746b42189751b2817f40a623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3da20f0055b74c0292fa05c61074e4d6" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 41319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "658a8336746b42189751b2817f40a623" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "68952289b9a242f7a23f07cbcb8b36f2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 45001 } health_report { overall_health: UNKNOWN } } }
I20260929 10:50:25.744947 28237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "39326fe5440f4d74abd39e9b375c9eaf" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: false } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: NON_VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: true } } }
I20260929 10:50:25.745065 28237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:25.745726 28239 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf: attempting to promote NON_VOTER 8d37f646d63549d480f2c2e479a0826c to VOTER
I20260929 10:50:25.745843 28239 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: false } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: false } }
I20260929 10:50:25.746212 27752 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [term 1 LEARNER]: Refusing update from remote peer 39326fe5440f4d74abd39e9b375c9eaf: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260929 10:50:25.746389 27563 raft_consensus.cc:1261] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [term 1 FOLLOWER]: Refusing update from remote peer 39326fe5440f4d74abd39e9b375c9eaf: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260929 10:50:25.746443 28237 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260929 10:50:25.746567 28239 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [LEADER]: Connected to new peer: Peer: permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260929 10:50:25.747049 28237 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 8d37f646d63549d480f2c2e479a0826c (127.20.80.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: false } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: false } } }
I20260929 10:50:25.747234 27563 raft_consensus.cc:2932] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 8d37f646d63549d480f2c2e479a0826c (127.20.80.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: false } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: false } } }
I20260929 10:50:25.747221 27752 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 8d37f646d63549d480f2c2e479a0826c (127.20.80.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: false } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: false } } }
I20260929 10:50:25.747997 28237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "39326fe5440f4d74abd39e9b375c9eaf" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: false } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: false } } }
I20260929 10:50:25.748085 28237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:25.748148 28237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "39326fe5440f4d74abd39e9b375c9eaf" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: false } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: false } } }
I20260929 10:50:25.748203 28237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:25.748157 27781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "39326fe5440f4d74abd39e9b375c9eaf" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: false } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: false } } }
I20260929 10:50:25.748245 27781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:50:25.748263 27704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "39326fe5440f4d74abd39e9b375c9eaf" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: false } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: false } } }
I20260929 10:50:25.748342 27704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:25.748416 28240 sys_catalog.cc:455] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "39326fe5440f4d74abd39e9b375c9eaf" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: false } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: false } } }
I20260929 10:50:25.748504 28240 sys_catalog.cc:458] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [sys.catalog]: This master's current role is: FOLLOWER
W20260929 10:50:25.759137 28199 tablet.cc:2369] T a66c8a22365e4e979d8ac59a7a63c90d P 658a8336746b42189751b2817f40a623: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:50:25.764146 28221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.761920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49198 (local address 127.20.80.195:35313)
0929 10:50:25.761981 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:25.761983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:25.761998 (+ 15us) server_negotiation.cc:407] Connection header received
0929 10:50:25.762031 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:25.762032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:25.762047 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:25.762116 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:50:25.762217 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.762682 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.763514 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.763707 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.763768 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.763891 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:50:25.763984 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:50:25.764045 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:25.764076 (+ 31us) server_negotiation.cc:299] Negotiation successful
0929 10:50:25.764095 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 10:50:25.765893 27875 raft_consensus.cc:1261] T a66c8a22365e4e979d8ac59a7a63c90d P 3da20f0055b74c0292fa05c61074e4d6 [term 1 FOLLOWER]: Refusing update from remote peer 658a8336746b42189751b2817f40a623: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:25.765899 28013 raft_consensus.cc:1261] T a66c8a22365e4e979d8ac59a7a63c90d P 68952289b9a242f7a23f07cbcb8b36f2 [term 1 FOLLOWER]: Refusing update from remote peer 658a8336746b42189751b2817f40a623: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:25.766273 28235 consensus_queue.cc:1035] T a66c8a22365e4e979d8ac59a7a63c90d P 658a8336746b42189751b2817f40a623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da20f0055b74c0292fa05c61074e4d6" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 41319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:50:25.766424 28229 consensus_queue.cc:1035] T a66c8a22365e4e979d8ac59a7a63c90d P 658a8336746b42189751b2817f40a623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68952289b9a242f7a23f07cbcb8b36f2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 45001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:50:25.768896 28242 mvcc.cc:205] Tried to move back new op lower bound from 7334621289534550016 to 7334621289445875712. Current Snapshot: MvccSnapshot[applied={T|T < 7334621289534550016}]
I20260929 10:50:25.769135 28244 mvcc.cc:205] Tried to move back new op lower bound from 7334621289534550016 to 7334621289445875712. Current Snapshot: MvccSnapshot[applied={T|T < 7334621289534550016}]
I20260929 10:50:25.772720 26950 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:50:25.772801 26950 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:50:25.774067 26950 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 39326fe5440f4d74abd39e9b375c9eaf: Sending DeleteTablet for 3 replicas of tablet a66c8a22365e4e979d8ac59a7a63c90d
I20260929 10:50:25.774360 27855 tablet_service.cc:1514] Processing DeleteTablet for tablet a66c8a22365e4e979d8ac59a7a63c90d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:25 UTC) from {username='oryx', principal='oryx/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:41932
I20260929 10:50:25.774406 28130 tablet_service.cc:1514] Processing DeleteTablet for tablet a66c8a22365e4e979d8ac59a7a63c90d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:25 UTC) from {username='oryx', principal='oryx/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:49190
I20260929 10:50:25.774624 28252 tablet_replica.cc:330] T a66c8a22365e4e979d8ac59a7a63c90d P 658a8336746b42189751b2817f40a623: stopping tablet replica
I20260929 10:50:25.774636 27993 tablet_service.cc:1514] Processing DeleteTablet for tablet a66c8a22365e4e979d8ac59a7a63c90d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:25 UTC) from {username='oryx', principal='oryx/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:53478
I20260929 10:50:25.774755 28252 raft_consensus.cc:2229] T a66c8a22365e4e979d8ac59a7a63c90d P 658a8336746b42189751b2817f40a623 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:50:25.774868 28252 raft_consensus.cc:2258] T a66c8a22365e4e979d8ac59a7a63c90d P 658a8336746b42189751b2817f40a623 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:25.774948 28253 tablet_replica.cc:330] T a66c8a22365e4e979d8ac59a7a63c90d P 68952289b9a242f7a23f07cbcb8b36f2: stopping tablet replica
I20260929 10:50:25.775050 28253 raft_consensus.cc:2229] T a66c8a22365e4e979d8ac59a7a63c90d P 68952289b9a242f7a23f07cbcb8b36f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:50:25.775157 28253 raft_consensus.cc:2258] T a66c8a22365e4e979d8ac59a7a63c90d P 68952289b9a242f7a23f07cbcb8b36f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:25.775594 28253 ts_tablet_manager.cc:1905] T a66c8a22365e4e979d8ac59a7a63c90d P 68952289b9a242f7a23f07cbcb8b36f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:25.776185 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 26918
I20260929 10:50:25.777928 28253 ts_tablet_manager.cc:1918] T a66c8a22365e4e979d8ac59a7a63c90d P 68952289b9a242f7a23f07cbcb8b36f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:50:25.778066 28253 log.cc:1196] T a66c8a22365e4e979d8ac59a7a63c90d P 68952289b9a242f7a23f07cbcb8b36f2: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-1/wal/wals/a66c8a22365e4e979d8ac59a7a63c90d
I20260929 10:50:25.778406 28253 ts_tablet_manager.cc:1939] T a66c8a22365e4e979d8ac59a7a63c90d P 68952289b9a242f7a23f07cbcb8b36f2: Deleting consensus metadata
W20260929 10:50:25.783004 27949 connection.cc:535] server connection from 127.0.0.1:53478 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 10:50:25.783752 27926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.775885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35858 (local address 127.20.80.252:39713)
0929 10:50:25.775951 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:25.775953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:25.775962 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:50:25.775983 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:25.775983 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:25.775994 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:25.776049 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:25.776753 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.777548 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.778617 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.778928 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.778971 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.780655 (+ 1684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:25.780656 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:25.780656 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:25.780663 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:25.782454 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.782860 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.782862 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.782862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.782919 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.783122 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.783123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.783124 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.783267 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:25.783345 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:25.783576 (+ 231us) server_negotiation.cc:299] Negotiation successful
0929 10:50:25.783656 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 10:50:25.785261 28252 ts_tablet_manager.cc:1905] T a66c8a22365e4e979d8ac59a7a63c90d P 658a8336746b42189751b2817f40a623: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:25.786583 28251 tablet_replica.cc:330] T a66c8a22365e4e979d8ac59a7a63c90d P 3da20f0055b74c0292fa05c61074e4d6: stopping tablet replica
I20260929 10:50:25.786741 28251 raft_consensus.cc:2229] T a66c8a22365e4e979d8ac59a7a63c90d P 3da20f0055b74c0292fa05c61074e4d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:50:25.786887 28251 raft_consensus.cc:2258] T a66c8a22365e4e979d8ac59a7a63c90d P 3da20f0055b74c0292fa05c61074e4d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:25.786892 28252 ts_tablet_manager.cc:1918] T a66c8a22365e4e979d8ac59a7a63c90d P 658a8336746b42189751b2817f40a623: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:50:25.787266 28252 log.cc:1196] T a66c8a22365e4e979d8ac59a7a63c90d P 658a8336746b42189751b2817f40a623: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-2/wal/wals/a66c8a22365e4e979d8ac59a7a63c90d
I20260929 10:50:25.787426 28251 ts_tablet_manager.cc:1905] T a66c8a22365e4e979d8ac59a7a63c90d P 3da20f0055b74c0292fa05c61074e4d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:25.787659 28252 ts_tablet_manager.cc:1939] T a66c8a22365e4e979d8ac59a7a63c90d P 658a8336746b42189751b2817f40a623: Deleting consensus metadata
W20260929 10:50:25.788084 28085 connection.cc:441] server connection from 127.0.0.1:49190 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49190 (request call id 1) could send its response
I20260929 10:50:25.789219 28251 ts_tablet_manager.cc:1918] T a66c8a22365e4e979d8ac59a7a63c90d P 3da20f0055b74c0292fa05c61074e4d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:50:25.789372 28251 log.cc:1196] T a66c8a22365e4e979d8ac59a7a63c90d P 3da20f0055b74c0292fa05c61074e4d6: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-0/wal/wals/a66c8a22365e4e979d8ac59a7a63c90d
I20260929 10:50:25.789741 28251 ts_tablet_manager.cc:1939] T a66c8a22365e4e979d8ac59a7a63c90d P 3da20f0055b74c0292fa05c61074e4d6: Deleting consensus metadata
W20260929 10:50:25.790758 27811 connection.cc:441] server connection from 127.0.0.1:41932 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41932 (request call id 1) could send its response
I20260929 10:50:25.793462 27926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.785034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35874 (local address 127.20.80.252:39713)
0929 10:50:25.785075 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:25.785077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:25.785379 (+ 302us) server_negotiation.cc:407] Connection header received
0929 10:50:25.785470 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:25.785471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:25.785480 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:25.785508 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:25.786569 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.787328 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.788463 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.788758 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.788798 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.790440 (+ 1642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:25.790441 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:25.790442 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:25.790448 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:25.792127 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.792546 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.792548 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.792548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.792600 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.792795 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.792796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.792796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.792912 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:25.793018 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:25.793254 (+ 236us) server_negotiation.cc:299] Negotiation successful
0929 10:50:25.793344 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 10:50:25.795540 27604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:25.785362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58112 (local address 127.20.80.253:44859)
0929 10:50:25.785410 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:25.785412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:25.787140 (+ 1728us) server_negotiation.cc:407] Connection header received
0929 10:50:25.789025 (+ 1885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:25.789026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:25.789039 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:25.789090 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:25.789803 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.790278 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.791181 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:25.791462 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:25.791527 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:25.792891 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:25.792893 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:25.792893 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:25.792900 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:25.794462 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.794803 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.794804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.794804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.794866 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:25.795060 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:25.795061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:25.795062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:25.795171 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:25.795268 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:25.795369 (+ 101us) server_negotiation.cc:299] Negotiation successful
0929 10:50:25.795443 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 10:50:26.770855 28261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:26.770580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.20.80.194:51429)
0929 10:50:26.770696 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:26.770770 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
I20260929 10:50:26.770892 28262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:26.770593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.20.80.193:35397)
0929 10:50:26.770714 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:26.770837 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":56,"threads_started":1}
W20260929 10:50:26.771057 28061 heartbeater.cc:644] Failed to heartbeat to 127.20.80.254:44567 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
W20260929 10:50:26.771057 27923 heartbeater.cc:644] Failed to heartbeat to 127.20.80.254:44567 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
I20260929 10:50:26.771413 28263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:26.771232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.20.80.195:47385)
0929 10:50:26.771315 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:26.771381 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69,"thread_start_us":47,"threads_started":1}
W20260929 10:50:26.771610 28197 heartbeater.cc:644] Failed to heartbeat to 127.20.80.254:44567 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
I20260929 10:50:27.274852 28264 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 39326fe5440f4d74abd39e9b375c9eaf)
I20260929 10:50:27.275027 28264 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: false } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: false } }
I20260929 10:50:27.275242 28265 raft_consensus.cc:482] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 39326fe5440f4d74abd39e9b375c9eaf)
I20260929 10:50:27.275328 28265 raft_consensus.cc:504] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: false } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: false } }
I20260929 10:50:27.275379 28264 leader_election.cc:288] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 39326fe5440f4d74abd39e9b375c9eaf (127.20.80.254:44567), da5626be71c64349953ab4a5ecb1d8db (127.20.80.253:44859)
I20260929 10:50:27.275550 28265 leader_election.cc:288] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 39326fe5440f4d74abd39e9b375c9eaf (127.20.80.254:44567), 8d37f646d63549d480f2c2e479a0826c (127.20.80.252:39713)
I20260929 10:50:27.275882 28266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:27.275499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.0.0.1:52976)
0929 10:50:27.275676 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:27.275832 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":49,"threads_started":1}
I20260929 10:50:27.275929 28270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:27.275682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.0.0.1:52986)
0929 10:50:27.275788 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:27.275884 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":82,"threads_started":1}
W20260929 10:50:27.276090 27537 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
W20260929 10:50:27.276110 27725 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
I20260929 10:50:27.276520 28266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:27.276440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.0.0.1:52988)
0929 10:50:27.276474 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:27.276495 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260929 10:50:27.276582 28270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:27.276499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.0.0.1:52998)
0929 10:50:27.276533 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:27.276556 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260929 10:50:27.276676 27725 leader_election.cc:334] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 39326fe5440f4d74abd39e9b375c9eaf (127.20.80.254:44567): Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
W20260929 10:50:27.276711 27537 leader_election.cc:334] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 39326fe5440f4d74abd39e9b375c9eaf (127.20.80.254:44567): Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
I20260929 10:50:27.278726 28271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:27.275815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35886 (local address 127.20.80.252:39713)
0929 10:50:27.275902 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:27.275904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:27.276070 (+ 166us) server_negotiation.cc:407] Connection header received
0929 10:50:27.276094 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:27.276095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:27.276105 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:27.276167 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:27.276350 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:27.277054 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:27.278020 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:27.278382 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:27.278466 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:27.278477 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:27.278620 (+ 143us) server_negotiation.cc:299] Negotiation successful
0929 10:50:27.278661 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71,"thread_start_us":48,"threads_started":1}
I20260929 10:50:27.278724 28269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:27.275712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:39713 (local address 127.0.0.1:35886)
0929 10:50:27.275946 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:27.275957 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:27.276016 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:27.276213 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:27.276214 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:27.276216 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:27.276270 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:27.276272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:27.277134 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:27.277135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:27.277936 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:27.277939 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:27.278466 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:27.278467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:27.278535 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:27.278538 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:50:27.278612 (+ 74us) client_negotiation.cc:240] Negotiation successful
0929 10:50:27.278655 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":33,"threads_started":1}
I20260929 10:50:27.278891 28267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:27.275537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:44859 (local address 127.0.0.1:58116)
0929 10:50:27.275813 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:27.275823 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:27.275907 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:27.276125 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:27.276126 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:27.276128 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:27.276184 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:27.276186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:27.277153 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:27.277154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:27.277999 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:27.278001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:27.278607 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:27.278608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:27.278675 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:27.278678 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:50:27.278743 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 10:50:27.278787 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":47,"threads_started":1}
I20260929 10:50:27.278965 28268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:27.275582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58116 (local address 127.20.80.253:44859)
0929 10:50:27.275736 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:27.275738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:27.275878 (+ 140us) server_negotiation.cc:407] Connection header received
0929 10:50:27.275999 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:27.276000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:27.276012 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:27.276052 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:27.276267 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:27.277065 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:27.278091 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:27.278527 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:27.278606 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:27.278617 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:27.278769 (+ 152us) server_negotiation.cc:299] Negotiation successful
0929 10:50:27.278886 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":35,"threads_started":1}
I20260929 10:50:27.279033 27752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "da5626be71c64349953ab4a5ecb1d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8d37f646d63549d480f2c2e479a0826c" is_pre_election: true
I20260929 10:50:27.279150 27752 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da5626be71c64349953ab4a5ecb1d8db in term 1.
I20260929 10:50:27.279254 27563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8d37f646d63549d480f2c2e479a0826c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "da5626be71c64349953ab4a5ecb1d8db" is_pre_election: true
I20260929 10:50:27.279330 27539 leader_election.cc:302] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8d37f646d63549d480f2c2e479a0826c, da5626be71c64349953ab4a5ecb1d8db; no voters: 39326fe5440f4d74abd39e9b375c9eaf
I20260929 10:50:27.279372 27563 raft_consensus.cc:2454] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d37f646d63549d480f2c2e479a0826c in term 1.
I20260929 10:50:27.279449 28265 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260929 10:50:27.279481 28265 raft_consensus.cc:482] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [term 1 FOLLOWER]: Starting leader election (detected failure of leader 39326fe5440f4d74abd39e9b375c9eaf)
I20260929 10:50:27.279485 27728 leader_election.cc:302] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8d37f646d63549d480f2c2e479a0826c, da5626be71c64349953ab4a5ecb1d8db; no voters: 39326fe5440f4d74abd39e9b375c9eaf
I20260929 10:50:27.279522 28265 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [term 1 FOLLOWER]: Advancing to term 2
I20260929 10:50:27.279590 28264 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260929 10:50:27.279628 28264 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 39326fe5440f4d74abd39e9b375c9eaf)
I20260929 10:50:27.279647 28264 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [term 1 FOLLOWER]: Advancing to term 2
I20260929 10:50:27.280259 28265 raft_consensus.cc:504] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: false } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: false } }
I20260929 10:50:27.280273 28264 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: false } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: false } }
I20260929 10:50:27.280400 28265 leader_election.cc:288] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [CANDIDATE]: Term 2 election: Requested vote from peers 39326fe5440f4d74abd39e9b375c9eaf (127.20.80.254:44567), 8d37f646d63549d480f2c2e479a0826c (127.20.80.252:39713)
I20260929 10:50:27.280462 28264 leader_election.cc:288] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [CANDIDATE]: Term 2 election: Requested vote from peers 39326fe5440f4d74abd39e9b375c9eaf (127.20.80.254:44567), da5626be71c64349953ab4a5ecb1d8db (127.20.80.253:44859)
I20260929 10:50:27.280562 27752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "da5626be71c64349953ab4a5ecb1d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8d37f646d63549d480f2c2e479a0826c"
I20260929 10:50:27.280619 28267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:27.280466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.0.0.1:53008)
0929 10:50:27.280572 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:27.280600 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260929 10:50:27.280616 27563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8d37f646d63549d480f2c2e479a0826c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "da5626be71c64349953ab4a5ecb1d8db"
I20260929 10:50:27.280658 27752 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate da5626be71c64349953ab4a5ecb1d8db in current term 2: Already voted for candidate 8d37f646d63549d480f2c2e479a0826c in this term.
I20260929 10:50:27.280704 27563 raft_consensus.cc:2379] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d37f646d63549d480f2c2e479a0826c in current term 2: Already voted for candidate da5626be71c64349953ab4a5ecb1d8db in this term.
I20260929 10:50:27.280786 28269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:27.280495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.0.0.1:53022)
0929 10:50:27.280730 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:27.280759 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260929 10:50:27.281117 28267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:27.281031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.0.0.1:53034)
0929 10:50:27.281067 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:27.281088 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260929 10:50:27.281172 28269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:27.281088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.0.0.1:53040)
0929 10:50:27.281120 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:27.281143 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260929 10:50:27.281189 27725 leader_election.cc:334] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 39326fe5440f4d74abd39e9b375c9eaf (127.20.80.254:44567): Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
I20260929 10:50:27.281221 27725 leader_election.cc:302] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8d37f646d63549d480f2c2e479a0826c; no voters: 39326fe5440f4d74abd39e9b375c9eaf, da5626be71c64349953ab4a5ecb1d8db
W20260929 10:50:27.281239 27537 leader_election.cc:334] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 39326fe5440f4d74abd39e9b375c9eaf (127.20.80.254:44567): Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
I20260929 10:50:27.281308 27537 leader_election.cc:302] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da5626be71c64349953ab4a5ecb1d8db; no voters: 39326fe5440f4d74abd39e9b375c9eaf, 8d37f646d63549d480f2c2e479a0826c
I20260929 10:50:27.281308 28264 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260929 10:50:27.281399 28265 raft_consensus.cc:2726] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260929 10:50:27.771616 28275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:27.771357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.20.80.194:44147)
0929 10:50:27.771518 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:27.771561 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":108,"threads_started":1}
I20260929 10:50:27.771615 28276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:27.771362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.20.80.193:33389)
0929 10:50:27.771532 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:27.771560 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":101,"threads_started":1}
I20260929 10:50:27.772104 28277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:27.771926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.20.80.195:60249)
0929 10:50:27.772030 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:27.772063 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":52,"threads_started":1}
I20260929 10:50:28.772308 28278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:28.772077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.20.80.193:51471)
0929 10:50:28.772220 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:28.772256 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":67,"threads_started":1}
I20260929 10:50:28.772460 28279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:28.772075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.20.80.194:44309)
0929 10:50:28.772220 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:28.772257 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":74,"threads_started":1}
I20260929 10:50:28.772642 28280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:28.772461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.20.80.195:59685)
0929 10:50:28.772563 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:28.772595 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":47,"threads_started":1}
I20260929 10:50:29.187620 28281 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:50:29.187727 28281 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: false } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: false } }
I20260929 10:50:29.187927 28281 leader_election.cc:288] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 39326fe5440f4d74abd39e9b375c9eaf (127.20.80.254:44567), da5626be71c64349953ab4a5ecb1d8db (127.20.80.253:44859)
I20260929 10:50:29.188138 27563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8d37f646d63549d480f2c2e479a0826c" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "da5626be71c64349953ab4a5ecb1d8db" is_pre_election: true
I20260929 10:50:29.188228 27563 raft_consensus.cc:2454] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d37f646d63549d480f2c2e479a0826c in term 2.
I20260929 10:50:29.188278 28282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:29.188032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.0.0.1:53268)
0929 10:50:29.188154 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:29.188188 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":74,"threads_started":1}
I20260929 10:50:29.188403 27728 leader_election.cc:302] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d37f646d63549d480f2c2e479a0826c, da5626be71c64349953ab4a5ecb1d8db; no voters:
I20260929 10:50:29.188501 28281 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260929 10:50:29.188544 28281 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:50:29.188558 28281 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [term 2 FOLLOWER]: Advancing to term 3
I20260929 10:50:29.188669 28282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:29.188583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.0.0.1:53274)
0929 10:50:29.188615 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:29.188641 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260929 10:50:29.188755 27725 leader_election.cc:334] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 39326fe5440f4d74abd39e9b375c9eaf (127.20.80.254:44567): Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
I20260929 10:50:29.189486 28281 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: false } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: false } }
I20260929 10:50:29.189621 28281 leader_election.cc:288] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [CANDIDATE]: Term 3 election: Requested vote from peers 39326fe5440f4d74abd39e9b375c9eaf (127.20.80.254:44567), da5626be71c64349953ab4a5ecb1d8db (127.20.80.253:44859)
I20260929 10:50:29.189776 28282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:29.189686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.0.0.1:53290)
0929 10:50:29.189720 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:29.189745 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260929 10:50:29.189775 27563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8d37f646d63549d480f2c2e479a0826c" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "da5626be71c64349953ab4a5ecb1d8db"
I20260929 10:50:29.189864 27563 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [term 2 FOLLOWER]: Advancing to term 3
I20260929 10:50:29.190114 28282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:29.190054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.0.0.1:53300)
0929 10:50:29.190079 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:29.190099 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260929 10:50:29.190208 27725 leader_election.cc:334] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 39326fe5440f4d74abd39e9b375c9eaf (127.20.80.254:44567): Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
I20260929 10:50:29.190608 27563 raft_consensus.cc:2454] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d37f646d63549d480f2c2e479a0826c in term 3.
I20260929 10:50:29.190763 27728 leader_election.cc:302] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8d37f646d63549d480f2c2e479a0826c, da5626be71c64349953ab4a5ecb1d8db; no voters: 39326fe5440f4d74abd39e9b375c9eaf
I20260929 10:50:29.190874 28281 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [term 3 FOLLOWER]: Leader election won for term 3
I20260929 10:50:29.191004 28281 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [term 3 LEADER]: Becoming Leader. State: Replica: 8d37f646d63549d480f2c2e479a0826c, State: Running, Role: LEADER
I20260929 10:50:29.191072 28281 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: false } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: false } }
I20260929 10:50:29.191425 28284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d37f646d63549d480f2c2e479a0826c. Latest consensus state: current_term: 3 leader_uuid: "8d37f646d63549d480f2c2e479a0826c" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: false } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: false } } }
I20260929 10:50:29.191525 28284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:29.191679 28286 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:50:29.192032 28286 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:50:29.192085 28286 catalog_manager.cc:1259] Loaded cluster ID: 3fef4c4dad1e479d988b0cb330c76f15
I20260929 10:50:29.192101 28286 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:50:29.192245 28286 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:50:29.192302 28286 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c: Loaded TSK: 0
I20260929 10:50:29.192413 28286 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:50:29.268473 27741 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35874:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:50:29.270157 27741 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:50:29.271210 28282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:29.271086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.0.0.1:53314)
0929 10:50:29.271148 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:29.271182 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260929 10:50:29.271229 27563 raft_consensus.cc:1261] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [term 3 FOLLOWER]: Refusing update from remote peer 8d37f646d63549d480f2c2e479a0826c: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260929 10:50:29.271443 28284 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [LEADER]: Connected to new peer: Peer: permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260929 10:50:29.271582 28282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:29.271486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.0.0.1:53328)
0929 10:50:29.271533 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:29.271557 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260929 10:50:29.271649 27725 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c -> Peer 39326fe5440f4d74abd39e9b375c9eaf (127.20.80.254:44567): Couldn't send request to peer 39326fe5440f4d74abd39e9b375c9eaf. Status: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260929 10:50:29.272071 28288 sys_catalog.cc:455] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d37f646d63549d480f2c2e479a0826c. Latest consensus state: current_term: 3 leader_uuid: "8d37f646d63549d480f2c2e479a0826c" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: false } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: false } } }
I20260929 10:50:29.272181 28288 sys_catalog.cc:458] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:50:29.272547 28287 mvcc.cc:205] Tried to move back new op lower bound from 7334621303892631552 to 7334621303567167488. Current Snapshot: MvccSnapshot[applied={T|T < 7334621303892631552}]
I20260929 10:50:29.272682 28281 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "8d37f646d63549d480f2c2e479a0826c" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: false } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: false } } }
I20260929 10:50:29.272794 28281 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:29.273108 28288 sys_catalog.cc:455] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "8d37f646d63549d480f2c2e479a0826c" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "39326fe5440f4d74abd39e9b375c9eaf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44567 } } peers { permanent_uuid: "da5626be71c64349953ab4a5ecb1d8db" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44859 } attrs { promote: false } } peers { permanent_uuid: "8d37f646d63549d480f2c2e479a0826c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 39713 } attrs { promote: false } } }
I20260929 10:50:29.273195 28288 sys_catalog.cc:458] T 00000000000000000000000000000000 P da5626be71c64349953ab4a5ecb1d8db [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:50:29.273247 28289 mvcc.cc:205] Tried to move back new op lower bound from 7334621303892631552 to 7334621303567167488. Current Snapshot: MvccSnapshot[applied={T|T < 7334621303892631552}]
I20260929 10:50:29.277855 28294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:29.274509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53482 (local address 127.20.80.194:45001)
0929 10:50:29.274780 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:29.274782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:29.274789 (+ 7us) server_negotiation.cc:407] Connection header received
0929 10:50:29.274815 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:29.274816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:29.274826 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:29.274918 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:29.275066 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:29.275858 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:29.276890 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:29.277523 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:29.277613 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:29.277627 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:29.277749 (+ 122us) server_negotiation.cc:299] Negotiation successful
0929 10:50:29.277790 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":63,"threads_started":1}
I20260929 10:50:29.277859 28282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:29.274452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:45001 (local address 127.0.0.1:53482)
0929 10:50:29.274482 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:29.274493 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:29.274544 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:29.274920 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:29.274921 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:29.274922 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:29.274977 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:29.274979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:29.275966 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:29.275968 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:29.276785 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:29.276787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:29.277614 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:29.277615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:29.277681 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:29.277686 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 10:50:29.277741 (+ 55us) client_negotiation.cc:240] Negotiation successful
0929 10:50:29.277785 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260929 10:50:29.278108 28295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:29.274541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:35313 (local address 127.0.0.1:49206)
0929 10:50:29.274645 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:29.274654 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:29.274691 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:29.275023 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:29.275024 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:29.275027 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:29.275080 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:29.275082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:29.275788 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:29.275789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:29.277013 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:29.277014 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:29.277896 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:29.277897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:29.277957 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:29.277961 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:50:29.277989 (+ 28us) client_negotiation.cc:240] Negotiation successful
0929 10:50:29.278031 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":2,"thread_start_us":61,"threads_started":1}
I20260929 10:50:29.278180 27993 tablet_service.cc:1467] Processing CreateTablet for tablet 211ddc0ab07b4b179e798c7f1b1bebc3 (DEFAULT_TABLE table=test-table [id=7f28972a03164e55818af95780df4e55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:29.278316 27993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 211ddc0ab07b4b179e798c7f1b1bebc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:29.278375 28293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:29.274497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:41319 (local address 127.0.0.1:41934)
0929 10:50:29.274626 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:29.274638 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:29.274691 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:29.274976 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:29.274977 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:29.274979 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:29.275073 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:29.275075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:29.275935 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:29.275936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:29.277011 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:29.277014 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:29.278102 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:29.278103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:29.278172 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:29.278176 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:50:29.278225 (+ 49us) client_negotiation.cc:240] Negotiation successful
0929 10:50:29.278318 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":69,"threads_started":1}
I20260929 10:50:29.278415 28297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:29.274791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49206 (local address 127.20.80.195:35313)
0929 10:50:29.274908 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:29.274910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:29.274918 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:50:29.274943 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:29.274943 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:29.274956 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:29.275007 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:29.275158 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:29.275701 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:29.277221 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:29.277784 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:29.278269 (+ 485us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:29.278285 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:29.278304 (+ 19us) server_negotiation.cc:299] Negotiation successful
0929 10:50:29.278345 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":49,"threads_started":1}
I20260929 10:50:29.278578 28296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:29.274676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41934 (local address 127.20.80.193:41319)
0929 10:50:29.274788 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:29.274790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:29.274799 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:50:29.274826 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:29.274826 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:29.274838 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:29.274933 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:29.275150 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:29.275864 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:29.277114 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:29.277844 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:29.277906 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:29.277918 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:29.278460 (+ 542us) server_negotiation.cc:299] Negotiation successful
0929 10:50:29.278512 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":70,"threads_started":1}
I20260929 10:50:29.278585 28130 tablet_service.cc:1467] Processing CreateTablet for tablet 211ddc0ab07b4b179e798c7f1b1bebc3 (DEFAULT_TABLE table=test-table [id=7f28972a03164e55818af95780df4e55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:29.278692 28130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 211ddc0ab07b4b179e798c7f1b1bebc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:29.278748 27855 tablet_service.cc:1467] Processing CreateTablet for tablet 211ddc0ab07b4b179e798c7f1b1bebc3 (DEFAULT_TABLE table=test-table [id=7f28972a03164e55818af95780df4e55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:29.278846 27855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 211ddc0ab07b4b179e798c7f1b1bebc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:29.280160 28299 tablet_bootstrap.cc:492] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 658a8336746b42189751b2817f40a623: Bootstrap starting.
I20260929 10:50:29.280251 28300 tablet_bootstrap.cc:492] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 3da20f0055b74c0292fa05c61074e4d6: Bootstrap starting.
I20260929 10:50:29.280558 28300 tablet_bootstrap.cc:654] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 3da20f0055b74c0292fa05c61074e4d6: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:29.280580 28299 tablet_bootstrap.cc:654] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 658a8336746b42189751b2817f40a623: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:29.280906 28298 tablet_bootstrap.cc:492] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 68952289b9a242f7a23f07cbcb8b36f2: Bootstrap starting.
I20260929 10:50:29.281131 28299 tablet_bootstrap.cc:492] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 658a8336746b42189751b2817f40a623: No bootstrap required, opened a new log
I20260929 10:50:29.281174 28299 ts_tablet_manager.cc:1397] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 658a8336746b42189751b2817f40a623: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:29.281283 28299 raft_consensus.cc:348] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 658a8336746b42189751b2817f40a623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68952289b9a242f7a23f07cbcb8b36f2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 45001 } } peers { permanent_uuid: "3da20f0055b74c0292fa05c61074e4d6" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 41319 } } peers { permanent_uuid: "658a8336746b42189751b2817f40a623" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35313 } }
I20260929 10:50:29.281358 28299 raft_consensus.cc:374] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 658a8336746b42189751b2817f40a623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:29.281374 28299 raft_consensus.cc:729] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 658a8336746b42189751b2817f40a623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 658a8336746b42189751b2817f40a623, State: Initialized, Role: FOLLOWER
I20260929 10:50:29.281392 28298 tablet_bootstrap.cc:654] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 68952289b9a242f7a23f07cbcb8b36f2: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:29.281424 28299 consensus_queue.cc:260] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 658a8336746b42189751b2817f40a623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68952289b9a242f7a23f07cbcb8b36f2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 45001 } } peers { permanent_uuid: "3da20f0055b74c0292fa05c61074e4d6" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 41319 } } peers { permanent_uuid: "658a8336746b42189751b2817f40a623" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35313 } }
I20260929 10:50:29.281553 28299 ts_tablet_manager.cc:1428] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 658a8336746b42189751b2817f40a623: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:29.281596 28300 tablet_bootstrap.cc:492] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 3da20f0055b74c0292fa05c61074e4d6: No bootstrap required, opened a new log
I20260929 10:50:29.281636 28300 ts_tablet_manager.cc:1397] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 3da20f0055b74c0292fa05c61074e4d6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:29.281845 28300 raft_consensus.cc:348] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 3da20f0055b74c0292fa05c61074e4d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68952289b9a242f7a23f07cbcb8b36f2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 45001 } } peers { permanent_uuid: "3da20f0055b74c0292fa05c61074e4d6" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 41319 } } peers { permanent_uuid: "658a8336746b42189751b2817f40a623" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35313 } }
I20260929 10:50:29.281922 28300 raft_consensus.cc:374] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 3da20f0055b74c0292fa05c61074e4d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:29.281967 28300 raft_consensus.cc:729] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 3da20f0055b74c0292fa05c61074e4d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da20f0055b74c0292fa05c61074e4d6, State: Initialized, Role: FOLLOWER
I20260929 10:50:29.282029 28300 consensus_queue.cc:260] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 3da20f0055b74c0292fa05c61074e4d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68952289b9a242f7a23f07cbcb8b36f2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 45001 } } peers { permanent_uuid: "3da20f0055b74c0292fa05c61074e4d6" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 41319 } } peers { permanent_uuid: "658a8336746b42189751b2817f40a623" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35313 } }
I20260929 10:50:29.282084 28196 heartbeater.cc:503] Master 127.20.80.252:39713 was elected leader, sending a full tablet report...
I20260929 10:50:29.282186 28300 ts_tablet_manager.cc:1428] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 3da20f0055b74c0292fa05c61074e4d6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:29.282815 27922 heartbeater.cc:503] Master 127.20.80.252:39713 was elected leader, sending a full tablet report...
I20260929 10:50:29.282994 28307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:29.282405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.20.80.193:33651)
0929 10:50:29.282915 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:29.282953 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":71,"threads_started":1}
I20260929 10:50:29.283315 28307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:29.283222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.20.80.193:50217)
0929 10:50:29.283253 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:29.283284 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260929 10:50:29.283501 28305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:29.281724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.20.80.195:51425)
0929 10:50:29.283397 (+ 1673us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:29.283469 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1651,"thread_start_us":99,"threads_started":1}
I20260929 10:50:29.283493 28298 tablet_bootstrap.cc:492] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 68952289b9a242f7a23f07cbcb8b36f2: No bootstrap required, opened a new log
I20260929 10:50:29.283560 28298 ts_tablet_manager.cc:1397] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 68952289b9a242f7a23f07cbcb8b36f2: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260929 10:50:29.283692 28298 raft_consensus.cc:348] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 68952289b9a242f7a23f07cbcb8b36f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68952289b9a242f7a23f07cbcb8b36f2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 45001 } } peers { permanent_uuid: "3da20f0055b74c0292fa05c61074e4d6" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 41319 } } peers { permanent_uuid: "658a8336746b42189751b2817f40a623" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35313 } }
I20260929 10:50:29.283790 28298 raft_consensus.cc:374] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 68952289b9a242f7a23f07cbcb8b36f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:29.283849 28298 raft_consensus.cc:729] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 68952289b9a242f7a23f07cbcb8b36f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68952289b9a242f7a23f07cbcb8b36f2, State: Initialized, Role: FOLLOWER
I20260929 10:50:29.283895 28305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:29.283749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.20.80.195:34047)
0929 10:50:29.283837 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:29.283868 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260929 10:50:29.283910 28298 consensus_queue.cc:260] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 68952289b9a242f7a23f07cbcb8b36f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68952289b9a242f7a23f07cbcb8b36f2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 45001 } } peers { permanent_uuid: "3da20f0055b74c0292fa05c61074e4d6" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 41319 } } peers { permanent_uuid: "658a8336746b42189751b2817f40a623" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35313 } }
I20260929 10:50:29.284083 28298 ts_tablet_manager.cc:1428] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 68952289b9a242f7a23f07cbcb8b36f2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:29.284432 28310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:29.284234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.20.80.194:49857)
0929 10:50:29.284359 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:29.284393 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":84,"threads_started":1}
I20260929 10:50:29.284499 28060 heartbeater.cc:503] Master 127.20.80.252:39713 was elected leader, sending a full tablet report...
I20260929 10:50:29.284734 28310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:29.284658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.20.80.194:52347)
0929 10:50:29.284691 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:29.284712 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260929 10:50:29.562451 28304 raft_consensus.cc:482] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 658a8336746b42189751b2817f40a623 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:50:29.562561 28304 raft_consensus.cc:504] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 658a8336746b42189751b2817f40a623 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68952289b9a242f7a23f07cbcb8b36f2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 45001 } } peers { permanent_uuid: "3da20f0055b74c0292fa05c61074e4d6" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 41319 } } peers { permanent_uuid: "658a8336746b42189751b2817f40a623" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35313 } }
I20260929 10:50:29.562811 28304 leader_election.cc:288] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 658a8336746b42189751b2817f40a623 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68952289b9a242f7a23f07cbcb8b36f2 (127.20.80.194:45001), 3da20f0055b74c0292fa05c61074e4d6 (127.20.80.193:41319)
I20260929 10:50:29.563031 28013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "211ddc0ab07b4b179e798c7f1b1bebc3" candidate_uuid: "658a8336746b42189751b2817f40a623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68952289b9a242f7a23f07cbcb8b36f2" is_pre_election: true
I20260929 10:50:29.563086 27875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "211ddc0ab07b4b179e798c7f1b1bebc3" candidate_uuid: "658a8336746b42189751b2817f40a623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da20f0055b74c0292fa05c61074e4d6" is_pre_election: true
I20260929 10:50:29.563160 28013 raft_consensus.cc:2454] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 68952289b9a242f7a23f07cbcb8b36f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 658a8336746b42189751b2817f40a623 in term 0.
I20260929 10:50:29.563163 27875 raft_consensus.cc:2454] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 3da20f0055b74c0292fa05c61074e4d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 658a8336746b42189751b2817f40a623 in term 0.
I20260929 10:50:29.563316 28085 leader_election.cc:302] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 658a8336746b42189751b2817f40a623 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 658a8336746b42189751b2817f40a623, 68952289b9a242f7a23f07cbcb8b36f2; no voters:
I20260929 10:50:29.563414 28304 raft_consensus.cc:2781] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 658a8336746b42189751b2817f40a623 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:50:29.563454 28304 raft_consensus.cc:482] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 658a8336746b42189751b2817f40a623 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:50:29.563485 28304 raft_consensus.cc:3037] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 658a8336746b42189751b2817f40a623 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:29.564258 28304 raft_consensus.cc:504] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 658a8336746b42189751b2817f40a623 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68952289b9a242f7a23f07cbcb8b36f2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 45001 } } peers { permanent_uuid: "3da20f0055b74c0292fa05c61074e4d6" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 41319 } } peers { permanent_uuid: "658a8336746b42189751b2817f40a623" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35313 } }
I20260929 10:50:29.564390 28304 leader_election.cc:288] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 658a8336746b42189751b2817f40a623 [CANDIDATE]: Term 1 election: Requested vote from peers 68952289b9a242f7a23f07cbcb8b36f2 (127.20.80.194:45001), 3da20f0055b74c0292fa05c61074e4d6 (127.20.80.193:41319)
I20260929 10:50:29.564522 28013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "211ddc0ab07b4b179e798c7f1b1bebc3" candidate_uuid: "658a8336746b42189751b2817f40a623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68952289b9a242f7a23f07cbcb8b36f2"
I20260929 10:50:29.564600 28013 raft_consensus.cc:3037] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 68952289b9a242f7a23f07cbcb8b36f2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:29.564568 27875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "211ddc0ab07b4b179e798c7f1b1bebc3" candidate_uuid: "658a8336746b42189751b2817f40a623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da20f0055b74c0292fa05c61074e4d6"
I20260929 10:50:29.564640 27875 raft_consensus.cc:3037] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 3da20f0055b74c0292fa05c61074e4d6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:29.565423 27875 raft_consensus.cc:2454] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 3da20f0055b74c0292fa05c61074e4d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658a8336746b42189751b2817f40a623 in term 1.
I20260929 10:50:29.565428 28013 raft_consensus.cc:2454] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 68952289b9a242f7a23f07cbcb8b36f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658a8336746b42189751b2817f40a623 in term 1.
I20260929 10:50:29.565588 28083 leader_election.cc:302] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 658a8336746b42189751b2817f40a623 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da20f0055b74c0292fa05c61074e4d6, 658a8336746b42189751b2817f40a623; no voters:
I20260929 10:50:29.565688 28304 raft_consensus.cc:2781] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 658a8336746b42189751b2817f40a623 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:29.565876 28304 raft_consensus.cc:686] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 658a8336746b42189751b2817f40a623 [term 1 LEADER]: Becoming Leader. State: Replica: 658a8336746b42189751b2817f40a623, State: Running, Role: LEADER
I20260929 10:50:29.565954 28304 consensus_queue.cc:237] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 658a8336746b42189751b2817f40a623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68952289b9a242f7a23f07cbcb8b36f2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 45001 } } peers { permanent_uuid: "3da20f0055b74c0292fa05c61074e4d6" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 41319 } } peers { permanent_uuid: "658a8336746b42189751b2817f40a623" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35313 } }
I20260929 10:50:29.566275 28305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:29.566122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.20.80.195:43511)
0929 10:50:29.566185 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:29.566243 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260929 10:50:29.566617 27741 catalog_manager.cc:5495] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 658a8336746b42189751b2817f40a623 reported cstate change: term changed from 0 to 1, leader changed from <none> to 658a8336746b42189751b2817f40a623 (127.20.80.195). New cstate: current_term: 1 leader_uuid: "658a8336746b42189751b2817f40a623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68952289b9a242f7a23f07cbcb8b36f2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 45001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da20f0055b74c0292fa05c61074e4d6" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 41319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "658a8336746b42189751b2817f40a623" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35313 } health_report { overall_health: HEALTHY } } }
I20260929 10:50:29.631281 28297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:29.629140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49208 (local address 127.20.80.195:35313)
0929 10:50:29.629189 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:29.629190 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:29.629200 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:50:29.629225 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:29.629226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:29.629235 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:29.629301 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:50:29.629430 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:29.629921 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:29.630634 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:29.630837 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:29.630919 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:29.631062 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:50:29.631093 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:50:29.631149 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:29.631198 (+ 49us) server_negotiation.cc:299] Negotiation successful
0929 10:50:29.631222 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 10:50:29.632277 27875 raft_consensus.cc:1261] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 3da20f0055b74c0292fa05c61074e4d6 [term 1 FOLLOWER]: Refusing update from remote peer 658a8336746b42189751b2817f40a623: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:29.632380 28013 raft_consensus.cc:1261] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 68952289b9a242f7a23f07cbcb8b36f2 [term 1 FOLLOWER]: Refusing update from remote peer 658a8336746b42189751b2817f40a623: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:29.632629 28311 consensus_queue.cc:1035] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 658a8336746b42189751b2817f40a623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da20f0055b74c0292fa05c61074e4d6" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 41319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:50:29.632747 28304 consensus_queue.cc:1035] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 658a8336746b42189751b2817f40a623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68952289b9a242f7a23f07cbcb8b36f2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 45001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:50:29.633426 28310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:29.633277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.20.80.194:43069)
0929 10:50:29.633343 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:29.633387 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260929 10:50:29.633615 28307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:29.633497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.20.80.193:50109)
0929 10:50:29.633545 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:29.633578 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260929 10:50:29.634676 28305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:29.634586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.20.80.195:45159)
0929 10:50:29.634624 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:29.634647 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260929 10:50:29.635111 28310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:29.635021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.20.80.194:46263)
0929 10:50:29.635055 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:29.635080 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260929 10:50:29.635270 28317 mvcc.cc:205] Tried to move back new op lower bound from 7334621305371959296 to 7334621305102581760. Current Snapshot: MvccSnapshot[applied={T|T < 7334621305371959296}]
I20260929 10:50:29.635759 28307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:29.635636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.20.80.193:49679)
0929 10:50:29.635706 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:29.635731 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260929 10:50:29.636168 28305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:29.636054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44567 (local address 127.20.80.195:60195)
0929 10:50:29.636103 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:29.636136 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260929 10:50:29.636138 27741 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:50:29.636221 27741 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:50:29.637555 27741 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8d37f646d63549d480f2c2e479a0826c: Sending DeleteTablet for 3 replicas of tablet 211ddc0ab07b4b179e798c7f1b1bebc3
I20260929 10:50:29.637846 27855 tablet_service.cc:1514] Processing DeleteTablet for tablet 211ddc0ab07b4b179e798c7f1b1bebc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:29 UTC) from {username='oryx', principal='oryx/127.20.80.252@KRBTEST.COM'} at 127.0.0.1:41934
I20260929 10:50:29.637867 27993 tablet_service.cc:1514] Processing DeleteTablet for tablet 211ddc0ab07b4b179e798c7f1b1bebc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:29 UTC) from {username='oryx', principal='oryx/127.20.80.252@KRBTEST.COM'} at 127.0.0.1:53482
I20260929 10:50:29.637912 28130 tablet_service.cc:1514] Processing DeleteTablet for tablet 211ddc0ab07b4b179e798c7f1b1bebc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:29 UTC) from {username='oryx', principal='oryx/127.20.80.252@KRBTEST.COM'} at 127.0.0.1:49206
I20260929 10:50:29.638087 28326 tablet_replica.cc:330] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 658a8336746b42189751b2817f40a623: stopping tablet replica
I20260929 10:50:29.638096 28325 tablet_replica.cc:330] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 68952289b9a242f7a23f07cbcb8b36f2: stopping tablet replica
I20260929 10:50:29.638151 28325 raft_consensus.cc:2229] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 68952289b9a242f7a23f07cbcb8b36f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:50:29.638156 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 27702
I20260929 10:50:29.638203 28325 raft_consensus.cc:2258] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 68952289b9a242f7a23f07cbcb8b36f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:29.638178 28326 raft_consensus.cc:2229] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 658a8336746b42189751b2817f40a623 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:50:29.640100 28326 raft_consensus.cc:2258] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 658a8336746b42189751b2817f40a623 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:29.640101 28325 ts_tablet_manager.cc:1905] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 68952289b9a242f7a23f07cbcb8b36f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:29.640434 28326 ts_tablet_manager.cc:1905] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 658a8336746b42189751b2817f40a623: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:29.642038 28325 ts_tablet_manager.cc:1918] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 68952289b9a242f7a23f07cbcb8b36f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:50:29.642102 28325 log.cc:1196] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 68952289b9a242f7a23f07cbcb8b36f2: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-1/wal/wals/211ddc0ab07b4b179e798c7f1b1bebc3
I20260929 10:50:29.642287 28326 ts_tablet_manager.cc:1918] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 658a8336746b42189751b2817f40a623: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:50:29.642347 28325 ts_tablet_manager.cc:1939] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 68952289b9a242f7a23f07cbcb8b36f2: Deleting consensus metadata
I20260929 10:50:29.642366 28326 log.cc:1196] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 658a8336746b42189751b2817f40a623: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790678996126930-20803-0/minicluster-data/ts-2/wal/wals/211ddc0ab07b4b179e798c7f1b1bebc3
I20260929 10:50:29.642606 28326 ts_tablet_manager.cc:1939] T 211ddc0ab07b4b179e798c7f1b1bebc3 P 658a8336746b42189751b2817f40a623: Deleting consensus metadata
I20260929 10:50:29.642962 27727 catalog_manager.cc:4841] TS 68952289b9a242f7a23f07cbcb8b36f2 (127.20.80.194:45001): tablet 211ddc0ab07b4b179e798c7f1b1bebc3 (table test-table [id=7f28972a03164e55818af95780df4e55]) successfully deleted
I20260929 10:50:29.643067 27727 catalog_manager.cc:4841] TS 658a8336746b42189751b2817f40a623 (127.20.80.195:35313): tablet 211ddc0ab07b4b179e798c7f1b1bebc3 (table test-table [id=7f28972a03164e55818af95780df4e55]) successfully deleted
I20260929 10:50:29.644367 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 27930
I20260929 10:50:29.644377 28293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:29.644246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:41319 (local address 127.0.0.1:41938)
0929 10:50:29.644307 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:29.644338 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.193:41319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260929 10:50:29.644526 27725 catalog_manager.cc:4584] TS 3da20f0055b74c0292fa05c61074e4d6 (127.20.80.193:41319): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 211ddc0ab07b4b179e798c7f1b1bebc3: Network error: Client connection negotiation failed: client connection to 127.20.80.193:41319: connect: Connection refused (error 111)
I20260929 10:50:29.649166 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 28067
I20260929 10:50:29.655093 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 27416
I20260929 10:50:29.660324 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 27607
2026-09-29T10:50:29Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6999 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:50:29 dist-test-slave-vqch krb5kdc[28329](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:50:29 dist-test-slave-vqch krb5kdc[28329](info): set up 2 sockets
Sep 29 10:50:29 dist-test-slave-vqch krb5kdc[28329](info): commencing operation
krb5kdc: starting...
W20260929 10:50:30.701197 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:50:30 dist-test-slave-vqch krb5kdc[28329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:50:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:50:30Z Disabled control of system clock
WARNING: no policy specified for oryx/127.20.80.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.80.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.80.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.254.keytab.
Entry for principal oryx/127.20.80.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.254.keytab.
I20260929 10:50:30.821233 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:42259
--webserver_interface=127.20.80.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.254.keytab
--principal=oryx/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:37707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:42259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:30.903069 28342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:30.903288 28342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:30.903311 28342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:30.904538 28342 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:50:30.904582 28342 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 10:50:30.904595 28342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:30.904608 28342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:30.904620 28342 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:50:30.904633 28342 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:50:30.906101 28342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:37707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:42259
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.254.keytab
--principal=oryx/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:42259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:30.906303 28342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:30.906473 28342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:30.911347 28348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:30.911378 28347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:30.911460 28350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:30.911521 28342 server_base.cc:1034] running on GCE node
I20260929 10:50:30.911842 28342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:30.912148 28342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:30.913300 28342 hybrid_clock.cc:648] HybridClock initialized: now 1790679030913270 us; error 44 us; skew 500 ppm
Sep 29 10:50:30 dist-test-slave-vqch krb5kdc[28329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679030, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:30.915867 28342 init.cc:377] Logged in from keytab as oryx/127.20.80.254@KRBTEST.COM (short username oryx)
I20260929 10:50:30.916356 28342 webserver.cc:460] Webserver started at http://127.20.80.254:38917/ using document root <none> and password file <none>
I20260929 10:50:30.916587 28342 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:30.916657 28342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:30.916811 28342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:30.917948 28342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "c04d4b30fd884a0f83bf748dc15c3a26"
format_stamp: "Formatted at 2026-09-29 10:50:30 on dist-test-slave-vqch"
server_key: "35755b520f3675d08d31abfae02e5b66"
server_key_iv: "a5bbb1221d229e70ae409229a7b15d81"
server_key_version: "encryptionkey@0"
I20260929 10:50:30.918331 28342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "c04d4b30fd884a0f83bf748dc15c3a26"
format_stamp: "Formatted at 2026-09-29 10:50:30 on dist-test-slave-vqch"
server_key: "35755b520f3675d08d31abfae02e5b66"
server_key_iv: "a5bbb1221d229e70ae409229a7b15d81"
server_key_version: "encryptionkey@0"
I20260929 10:50:30.919747 28342 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:50:30.920529 28357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:30.920710 28342 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:30.920817 28342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "c04d4b30fd884a0f83bf748dc15c3a26"
format_stamp: "Formatted at 2026-09-29 10:50:30 on dist-test-slave-vqch"
server_key: "35755b520f3675d08d31abfae02e5b66"
server_key_iv: "a5bbb1221d229e70ae409229a7b15d81"
server_key_version: "encryptionkey@0"
I20260929 10:50:30.920913 28342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:30.942905 28342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:30.944937 28342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:30.945147 28342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:30.950098 28342 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:42259
I20260929 10:50:30.950165 28420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:42259 every 8 connection(s)
I20260929 10:50:30.950536 28342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:50:30.951251 28421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:30.953553 28421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c04d4b30fd884a0f83bf748dc15c3a26: Bootstrap starting.
I20260929 10:50:30.954104 28421 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c04d4b30fd884a0f83bf748dc15c3a26: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:30.954361 28421 log.cc:824] T 00000000000000000000000000000000 P c04d4b30fd884a0f83bf748dc15c3a26: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:30.954977 28421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c04d4b30fd884a0f83bf748dc15c3a26: No bootstrap required, opened a new log
I20260929 10:50:30.956641 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 28342
I20260929 10:50:30.956782 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:50:30.957067 20803 external_mini_cluster.cc:1428] Setting key 1f5f7178251c5ffaa71b81d0ca04714c
I20260929 10:50:30.957485 28421 raft_consensus.cc:348] T 00000000000000000000000000000000 P c04d4b30fd884a0f83bf748dc15c3a26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c04d4b30fd884a0f83bf748dc15c3a26" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 42259 } }
I20260929 10:50:30.957671 28421 raft_consensus.cc:374] T 00000000000000000000000000000000 P c04d4b30fd884a0f83bf748dc15c3a26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:30.957749 28421 raft_consensus.cc:729] T 00000000000000000000000000000000 P c04d4b30fd884a0f83bf748dc15c3a26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c04d4b30fd884a0f83bf748dc15c3a26, State: Initialized, Role: FOLLOWER
I20260929 10:50:30.957898 28421 consensus_queue.cc:260] T 00000000000000000000000000000000 P c04d4b30fd884a0f83bf748dc15c3a26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c04d4b30fd884a0f83bf748dc15c3a26" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 42259 } }
I20260929 10:50:30.957980 28421 raft_consensus.cc:388] T 00000000000000000000000000000000 P c04d4b30fd884a0f83bf748dc15c3a26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:50:30.958003 28421 raft_consensus.cc:482] T 00000000000000000000000000000000 P c04d4b30fd884a0f83bf748dc15c3a26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:50:30.958070 28421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c04d4b30fd884a0f83bf748dc15c3a26 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:30.958626 28421 raft_consensus.cc:504] T 00000000000000000000000000000000 P c04d4b30fd884a0f83bf748dc15c3a26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c04d4b30fd884a0f83bf748dc15c3a26" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 42259 } }
I20260929 10:50:30.958745 28421 leader_election.cc:302] T 00000000000000000000000000000000 P c04d4b30fd884a0f83bf748dc15c3a26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c04d4b30fd884a0f83bf748dc15c3a26; no voters:
I20260929 10:50:30.958899 28421 leader_election.cc:288] T 00000000000000000000000000000000 P c04d4b30fd884a0f83bf748dc15c3a26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:50:30.958941 28426 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c04d4b30fd884a0f83bf748dc15c3a26 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:30.959113 28426 raft_consensus.cc:686] T 00000000000000000000000000000000 P c04d4b30fd884a0f83bf748dc15c3a26 [term 1 LEADER]: Becoming Leader. State: Replica: c04d4b30fd884a0f83bf748dc15c3a26, State: Running, Role: LEADER
I20260929 10:50:30.959146 28421 sys_catalog.cc:564] T 00000000000000000000000000000000 P c04d4b30fd884a0f83bf748dc15c3a26 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:50:30.959296 28426 consensus_queue.cc:237] T 00000000000000000000000000000000 P c04d4b30fd884a0f83bf748dc15c3a26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c04d4b30fd884a0f83bf748dc15c3a26" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 42259 } }
I20260929 10:50:30.959712 28426 sys_catalog.cc:455] T 00000000000000000000000000000000 P c04d4b30fd884a0f83bf748dc15c3a26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c04d4b30fd884a0f83bf748dc15c3a26. Latest consensus state: current_term: 1 leader_uuid: "c04d4b30fd884a0f83bf748dc15c3a26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c04d4b30fd884a0f83bf748dc15c3a26" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 42259 } } }
I20260929 10:50:30.959803 28426 sys_catalog.cc:458] T 00000000000000000000000000000000 P c04d4b30fd884a0f83bf748dc15c3a26 [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:30.959721 28427 sys_catalog.cc:455] T 00000000000000000000000000000000 P c04d4b30fd884a0f83bf748dc15c3a26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c04d4b30fd884a0f83bf748dc15c3a26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c04d4b30fd884a0f83bf748dc15c3a26" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 42259 } } }
I20260929 10:50:30.959892 28427 sys_catalog.cc:458] T 00000000000000000000000000000000 P c04d4b30fd884a0f83bf748dc15c3a26 [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:30.960014 28435 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:50:30.960521 28435 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:50:30.962162 28435 catalog_manager.cc:1347] Generated new cluster ID: d75eef1773a94de8824aa63d9dfbff20
I20260929 10:50:30.962215 28435 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 10:50:30 dist-test-slave-vqch krb5kdc[28329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM
I20260929 10:50:30.967682 28435 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:50:30.968096 28425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:30.957646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57628 (local address 127.20.80.254:42259)
0929 10:50:30.957820 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:30.957831 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:30.957865 (+ 34us) server_negotiation.cc:407] Connection header received
0929 10:50:30.958018 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:30.958026 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:30.958086 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:30.958251 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:30.958902 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:30.960450 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:30.961596 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:30.962174 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:30.962620 (+ 446us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:30.964658 (+ 2038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:30.964662 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:30.964663 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:30.964682 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:30.966397 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:30.966772 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:30.966776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:30.966778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:30.966878 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:30.967088 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:30.967089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:30.967090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:30.967380 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:30.967560 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:30.967777 (+ 217us) server_negotiation.cc:299] Negotiation successful
0929 10:50:30.967926 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":88,"threads_started":1}
I20260929 10:50:30.968436 28435 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:50:30.973204 28435 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c04d4b30fd884a0f83bf748dc15c3a26: Generated new TSK 0
I20260929 10:50:30.973372 28435 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.20.80.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.80.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.80.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.193.keytab.
Entry for principal oryx/127.20.80.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.193.keytab.
I20260929 10:50:31.008590 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.193:0
--local_ip_for_outbound_sockets=127.20.80.193
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:42259
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.193.keytab
--principal=oryx/127.20.80.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:37707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:31.088447 28447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:31.088677 28447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:31.088716 28447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:31.089902 28447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:31.089967 28447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:31.090003 28447 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 10:50:31.090056 28447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.193
I20260929 10:50:31.091559 28447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:37707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.193.keytab
--principal=oryx/127.20.80.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.80.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.80.254:42259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.193
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:31.091825 28447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:31.092065 28447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:31.097621 28454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:31.097730 28456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:31.098034 28447 server_base.cc:1034] running on GCE node
W20260929 10:50:31.098186 28453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:31.098415 28447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:31.098614 28447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:31.099753 28447 hybrid_clock.cc:648] HybridClock initialized: now 1790679031099732 us; error 35 us; skew 500 ppm
Sep 29 10:50:31 dist-test-slave-vqch krb5kdc[28329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679031, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:31.102247 28447 init.cc:377] Logged in from keytab as oryx/127.20.80.193@KRBTEST.COM (short username oryx)
I20260929 10:50:31.102676 28447 webserver.cc:460] Webserver started at http://127.20.80.193:39011/ using document root <none> and password file <none>
I20260929 10:50:31.102922 28447 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:31.102991 28447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:31.103178 28447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:31.104254 28447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-0/data/instance:
uuid: "afa84438b9224029840033fd30444c3f"
format_stamp: "Formatted at 2026-09-29 10:50:31 on dist-test-slave-vqch"
server_key: "c39a0102a6765c8eae74e16bf1f44eba"
server_key_iv: "0a8aa6e5279f58a8e7d59cb05f5117ff"
server_key_version: "encryptionkey@0"
I20260929 10:50:31.104544 28447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance:
uuid: "afa84438b9224029840033fd30444c3f"
format_stamp: "Formatted at 2026-09-29 10:50:31 on dist-test-slave-vqch"
server_key: "c39a0102a6765c8eae74e16bf1f44eba"
server_key_iv: "0a8aa6e5279f58a8e7d59cb05f5117ff"
server_key_version: "encryptionkey@0"
I20260929 10:50:31.105837 28447 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:50:31.106494 28463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:31.106674 28447 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:31.106738 28447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-0/wal
uuid: "afa84438b9224029840033fd30444c3f"
format_stamp: "Formatted at 2026-09-29 10:50:31 on dist-test-slave-vqch"
server_key: "c39a0102a6765c8eae74e16bf1f44eba"
server_key_iv: "0a8aa6e5279f58a8e7d59cb05f5117ff"
server_key_version: "encryptionkey@0"
I20260929 10:50:31.106787 28447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:31.118860 28447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:31.120571 28447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:31.120680 28447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:31.121250 28447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:31.121292 28447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:31.121315 28447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:31.121400 28447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 10:50:31 dist-test-slave-vqch krb5kdc[28329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679031, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM
I20260929 10:50:31.128587 28447 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.193:34221
I20260929 10:50:31.128672 28578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.193:34221 every 8 connection(s)
I20260929 10:50:31.129014 28447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
I20260929 10:50:31.131304 28425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:31.121629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:45397 (local address 127.20.80.254:42259)
0929 10:50:31.121670 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:31.121672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:31.121879 (+ 207us) server_negotiation.cc:407] Connection header received
0929 10:50:31.122401 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:31.122403 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:31.122416 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:31.122496 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:31.123357 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.124206 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.125848 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.126224 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.126318 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:31.128619 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:31.128620 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:31.128620 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:31.128627 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:31.130232 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:31.130610 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:31.130610 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:31.130611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:31.130669 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:31.130851 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:31.130852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:31.130852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:31.130951 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:31.130998 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:31.131173 (+ 175us) server_negotiation.cc:299] Negotiation successful
0929 10:50:31.131241 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 10:50:31.131542 28472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:31.121627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:42259 (local address 127.20.80.193:45397)
0929 10:50:31.121765 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:31.121799 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:31.122262 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:31.122549 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:31.122552 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:31.122878 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:31.123180 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:31.123192 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.124305 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.124306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:31.125748 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:31.125751 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.126357 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.126358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:31.126492 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:31.127015 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:31.127023 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:31.128495 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:31.130302 (+ 1807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:31.130308 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:31.130310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:31.130536 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:31.130716 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:31.130717 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:31.130717 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:31.130802 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:31.130988 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:31.130990 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:31.131089 (+ 99us) client_negotiation.cc:769] Sending connection context
0929 10:50:31.131163 (+ 74us) client_negotiation.cc:240] Negotiation successful
0929 10:50:31.131277 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":71,"threads_started":1}
I20260929 10:50:31.132067 28579 heartbeater.cc:347] Connected to a master server at 127.20.80.254:42259
I20260929 10:50:31.132171 28579 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:31.132339 28579 heartbeater.cc:511] Master 127.20.80.254:42259 requested a full tablet report, sending...
I20260929 10:50:31.132954 28375 ts_manager.cc:194] Registered new tserver with Master: afa84438b9224029840033fd30444c3f (127.20.80.193:34221)
I20260929 10:50:31.133668 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 28447
I20260929 10:50:31.133788 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance
I20260929 10:50:31.133930 28375 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.80.193@KRBTEST.COM'} at 127.20.80.193:45397
I20260929 10:50:31.134028 20803 external_mini_cluster.cc:1428] Setting key e9b02b288c5c76a4845ecb41dbde6490
I20260929 10:50:31.139058 28425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:31.133174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:53527 (local address 127.20.80.254:42259)
0929 10:50:31.133217 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:31.133219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:31.133320 (+ 101us) server_negotiation.cc:407] Connection header received
0929 10:50:31.133387 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:31.133388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:31.133396 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:31.133435 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:31.133778 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.134463 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.135230 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.135535 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.135608 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:31.136587 (+ 979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:31.136588 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:31.136588 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:31.136594 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:31.137984 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:31.138377 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:31.138378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:31.138378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:31.138426 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:31.138630 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:31.138631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:31.138631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:31.138721 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:31.138769 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:31.138944 (+ 175us) server_negotiation.cc:299] Negotiation successful
0929 10:50:31.139002 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 10:50:31.139189 28585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:31.133137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:42259 (local address 127.20.80.193:53527)
0929 10:50:31.133263 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:31.133273 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:31.133332 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:31.133447 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:31.133447 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:31.133644 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:31.133698 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:31.133700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.134581 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.134582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:31.135145 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:31.135147 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.135611 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.135611 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:31.135663 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:31.136098 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:31.136104 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:31.136504 (+ 400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:31.138065 (+ 1561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:31.138066 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:31.138067 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:31.138299 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:31.138499 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:31.138500 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:31.138501 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:31.138568 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:31.138784 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:31.138785 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:31.138836 (+ 51us) client_negotiation.cc:769] Sending connection context
0929 10:50:31.138920 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 10:50:31.139033 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":11,"thread_start_us":73,"threads_started":1}
WARNING: no policy specified for oryx/127.20.80.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.80.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.80.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.194.keytab.
Entry for principal oryx/127.20.80.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.194.keytab.
I20260929 10:50:31.159713 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.194:0
--local_ip_for_outbound_sockets=127.20.80.194
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:42259
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.194.keytab
--principal=oryx/127.20.80.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:37707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:31.240200 28588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:31.240389 28588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:31.240579 28588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:31.241992 28588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:31.242035 28588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:31.242056 28588 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 10:50:31.242074 28588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.194
I20260929 10:50:31.243589 28588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:37707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.194.keytab
--principal=oryx/127.20.80.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.80.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.80.254:42259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.194
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:31.243836 28588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:31.244087 28588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:31.249042 28595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:31.249106 28597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:31.249048 28594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:31.249293 28588 server_base.cc:1034] running on GCE node
I20260929 10:50:31.249543 28588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:31.249770 28588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:31.250934 28588 hybrid_clock.cc:648] HybridClock initialized: now 1790679031250932 us; error 39 us; skew 500 ppm
Sep 29 10:50:31 dist-test-slave-vqch krb5kdc[28329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679031, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:31.253492 28588 init.cc:377] Logged in from keytab as oryx/127.20.80.194@KRBTEST.COM (short username oryx)
I20260929 10:50:31.253939 28588 webserver.cc:460] Webserver started at http://127.20.80.194:34199/ using document root <none> and password file <none>
I20260929 10:50:31.254181 28588 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:31.254251 28588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:31.254403 28588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:31.255429 28588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-1/data/instance:
uuid: "db5e4a9496764d8d9a2f9bf9fa9df507"
format_stamp: "Formatted at 2026-09-29 10:50:31 on dist-test-slave-vqch"
server_key: "6ab22d2b0a9e3d8590bfa5913039dd6d"
server_key_iv: "e6d89162327de1337fd014681415fac7"
server_key_version: "encryptionkey@0"
I20260929 10:50:31.255791 28588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance:
uuid: "db5e4a9496764d8d9a2f9bf9fa9df507"
format_stamp: "Formatted at 2026-09-29 10:50:31 on dist-test-slave-vqch"
server_key: "6ab22d2b0a9e3d8590bfa5913039dd6d"
server_key_iv: "e6d89162327de1337fd014681415fac7"
server_key_version: "encryptionkey@0"
I20260929 10:50:31.257256 28588 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 10:50:31.258041 28604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:31.258224 28588 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:31.258325 28588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-1/wal
uuid: "db5e4a9496764d8d9a2f9bf9fa9df507"
format_stamp: "Formatted at 2026-09-29 10:50:31 on dist-test-slave-vqch"
server_key: "6ab22d2b0a9e3d8590bfa5913039dd6d"
server_key_iv: "e6d89162327de1337fd014681415fac7"
server_key_version: "encryptionkey@0"
I20260929 10:50:31.258406 28588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:31.276022 28588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:31.277958 28588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:31.278124 28588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:31.278652 28588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:31.278717 28588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:31.278765 28588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:31.278792 28588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 10:50:31 dist-test-slave-vqch krb5kdc[28329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679031, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM
I20260929 10:50:31.285920 28588 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.194:33969
I20260929 10:50:31.286013 28719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.194:33969 every 8 connection(s)
I20260929 10:50:31.286334 28588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
I20260929 10:50:31.288550 28425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:31.279181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:43929 (local address 127.20.80.254:42259)
0929 10:50:31.279256 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:31.279259 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:31.279427 (+ 168us) server_negotiation.cc:407] Connection header received
0929 10:50:31.279908 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:31.279910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:31.279924 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:31.280005 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:31.280891 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.281829 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.282907 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.283159 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.283222 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:31.285638 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:31.285640 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:31.285641 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:31.285649 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:31.287107 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:31.287551 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:31.287553 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:31.287553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:31.287616 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:31.287889 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:31.287891 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:31.287891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:31.288025 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:31.288109 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:31.288380 (+ 271us) server_negotiation.cc:299] Negotiation successful
0929 10:50:31.288460 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 10:50:31.288729 28613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:31.279134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:42259 (local address 127.20.80.194:43929)
0929 10:50:31.279296 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:31.279336 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:31.279811 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:31.280073 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:31.280076 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:31.280478 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:31.280773 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:31.280781 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.281932 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.281933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:31.282802 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:31.282804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.283264 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.283266 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:31.283390 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:31.283973 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:31.283995 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:31.285542 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:31.287204 (+ 1662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:31.287212 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:31.287214 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:31.287470 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:31.287684 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:31.287685 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:31.287686 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:31.287802 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:31.288100 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:31.288106 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:31.288245 (+ 139us) client_negotiation.cc:769] Sending connection context
0929 10:50:31.288353 (+ 108us) client_negotiation.cc:240] Negotiation successful
0929 10:50:31.288509 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":78,"threads_started":1}
I20260929 10:50:31.289268 28720 heartbeater.cc:347] Connected to a master server at 127.20.80.254:42259
I20260929 10:50:31.289350 28720 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:31.289548 28720 heartbeater.cc:511] Master 127.20.80.254:42259 requested a full tablet report, sending...
I20260929 10:50:31.289947 28375 ts_manager.cc:194] Registered new tserver with Master: db5e4a9496764d8d9a2f9bf9fa9df507 (127.20.80.194:33969)
I20260929 10:50:31.290372 28375 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.80.194@KRBTEST.COM'} at 127.20.80.194:43929
I20260929 10:50:31.294843 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 28588
I20260929 10:50:31.294975 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance
I20260929 10:50:31.295198 20803 external_mini_cluster.cc:1428] Setting key 4098070120b417afba958fbb1a13f747
I20260929 10:50:31.295889 28425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:31.290387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:44031 (local address 127.20.80.254:42259)
0929 10:50:31.290430 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:31.290431 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:31.290559 (+ 128us) server_negotiation.cc:407] Connection header received
0929 10:50:31.290629 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:31.290630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:31.290638 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:31.290695 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:31.290999 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.291497 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.292300 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.292483 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.292533 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:31.293553 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:31.293554 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:31.293554 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:31.293559 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:31.294689 (+ 1130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:31.295024 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:31.295025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:31.295026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:31.295084 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:31.295279 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:31.295281 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:31.295281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:31.295459 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:31.295543 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:31.295720 (+ 177us) server_negotiation.cc:299] Negotiation successful
0929 10:50:31.295792 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 10:50:31.295898 28726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:31.290304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:42259 (local address 127.20.80.194:44031)
0929 10:50:31.290497 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:31.290509 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:31.290574 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:31.290697 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:31.290698 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:31.290871 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:31.290927 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:31.290928 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.291706 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.291707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:31.292229 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:31.292231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.292571 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.292572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:31.292630 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:31.293127 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:31.293133 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:31.293478 (+ 345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:31.294768 (+ 1290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:31.294769 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:31.294769 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:31.294957 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:31.295156 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:31.295157 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:31.295157 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:31.295211 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:31.295537 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:31.295538 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:31.295587 (+ 49us) client_negotiation.cc:769] Sending connection context
0929 10:50:31.295690 (+ 103us) client_negotiation.cc:240] Negotiation successful
0929 10:50:31.295794 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":84,"threads_started":1}
WARNING: no policy specified for oryx/127.20.80.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.80.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.80.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.195.keytab.
Entry for principal oryx/127.20.80.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.195.keytab.
I20260929 10:50:31.320420 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.195:0
--local_ip_for_outbound_sockets=127.20.80.195
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:42259
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.195.keytab
--principal=oryx/127.20.80.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:37707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:31.400729 28729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:31.400938 28729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:31.400960 28729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:31.402132 28729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:31.402173 28729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:31.402194 28729 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 10:50:31.402210 28729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.195
I20260929 10:50:31.403685 28729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:37707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.195.keytab
--principal=oryx/127.20.80.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.80.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.80.254:42259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.195
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:31.403975 28729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:31.404160 28729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:31.409114 28734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:31.409135 28736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:31.409142 28729 server_base.cc:1034] running on GCE node
W20260929 10:50:31.409111 28738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:31.409608 28729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:31.409881 28729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:31.411046 28729 hybrid_clock.cc:648] HybridClock initialized: now 1790679031411044 us; error 35 us; skew 500 ppm
Sep 29 10:50:31 dist-test-slave-vqch krb5kdc[28329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679031, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:31.413611 28729 init.cc:377] Logged in from keytab as oryx/127.20.80.195@KRBTEST.COM (short username oryx)
I20260929 10:50:31.414079 28729 webserver.cc:460] Webserver started at http://127.20.80.195:45601/ using document root <none> and password file <none>
I20260929 10:50:31.414315 28729 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:31.414386 28729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:31.414518 28729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:31.415596 28729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-2/data/instance:
uuid: "97c90aace77b4f56b9f8224b3688aba4"
format_stamp: "Formatted at 2026-09-29 10:50:31 on dist-test-slave-vqch"
server_key: "a8197239ab5654b7ad7c116f46314361"
server_key_iv: "6c975f6b0a7aa0c5697f134e6eaf750a"
server_key_version: "encryptionkey@0"
I20260929 10:50:31.415956 28729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance:
uuid: "97c90aace77b4f56b9f8224b3688aba4"
format_stamp: "Formatted at 2026-09-29 10:50:31 on dist-test-slave-vqch"
server_key: "a8197239ab5654b7ad7c116f46314361"
server_key_iv: "6c975f6b0a7aa0c5697f134e6eaf750a"
server_key_version: "encryptionkey@0"
I20260929 10:50:31.417428 28729 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:50:31.418290 28745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:31.418463 28729 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:31.418557 28729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-2/wal
uuid: "97c90aace77b4f56b9f8224b3688aba4"
format_stamp: "Formatted at 2026-09-29 10:50:31 on dist-test-slave-vqch"
server_key: "a8197239ab5654b7ad7c116f46314361"
server_key_iv: "6c975f6b0a7aa0c5697f134e6eaf750a"
server_key_version: "encryptionkey@0"
I20260929 10:50:31.418637 28729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:31.437562 28729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:31.439610 28729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:31.439793 28729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:31.440446 28729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:31.440497 28729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:31.440544 28729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:31.440559 28729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 10:50:31 dist-test-slave-vqch krb5kdc[28329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679031, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM
I20260929 10:50:31.448212 28729 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.195:41667
I20260929 10:50:31.448277 28860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.195:41667 every 8 connection(s)
I20260929 10:50:31.448575 28729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
I20260929 10:50:31.451169 28425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:31.440923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:35261 (local address 127.20.80.254:42259)
0929 10:50:31.441043 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:31.441046 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:31.441372 (+ 326us) server_negotiation.cc:407] Connection header received
0929 10:50:31.442384 (+ 1012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:31.442387 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:31.442411 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:31.442669 (+ 258us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:31.443627 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.444189 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.445951 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.446228 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.446324 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:31.448733 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:31.448735 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:31.448735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:31.448742 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:31.450028 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:31.450405 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:31.450406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:31.450406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:31.450456 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:31.450664 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:31.450665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:31.450665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:31.450769 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:31.450820 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:31.451045 (+ 225us) server_negotiation.cc:299] Negotiation successful
0929 10:50:31.451107 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260929 10:50:31.451344 28754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:31.440915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:42259 (local address 127.20.80.195:35261)
0929 10:50:31.441187 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:31.441245 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:31.442082 (+ 837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:31.442748 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:31.442754 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:31.443181 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:31.443516 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:31.443524 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.444268 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.444269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:31.445852 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:31.445856 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.446337 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.446338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:31.446493 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:31.447067 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:31.447077 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:31.448595 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:31.450105 (+ 1510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:31.450110 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:31.450111 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:31.450335 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:31.450516 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:31.450517 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:31.450517 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:31.450610 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:31.450822 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:31.450826 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:31.450935 (+ 109us) client_negotiation.cc:769] Sending connection context
0929 10:50:31.451016 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 10:50:31.451132 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":120,"threads_started":1}
I20260929 10:50:31.451817 28861 heartbeater.cc:347] Connected to a master server at 127.20.80.254:42259
I20260929 10:50:31.451905 28861 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:31.452099 28861 heartbeater.cc:511] Master 127.20.80.254:42259 requested a full tablet report, sending...
I20260929 10:50:31.452565 28373 ts_manager.cc:194] Registered new tserver with Master: 97c90aace77b4f56b9f8224b3688aba4 (127.20.80.195:41667)
I20260929 10:50:31.453096 28373 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.80.195@KRBTEST.COM'} at 127.20.80.195:35261
I20260929 10:50:31.455503 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 28729
I20260929 10:50:31.455636 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance
I20260929 10:50:31.455868 20803 external_mini_cluster.cc:1428] Setting key 82335813817c7e9d87563b456c1b694b
I20260929 10:50:31.456804 20803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:50:31.459069 28867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:31.452909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:42259 (local address 127.20.80.195:53433)
0929 10:50:31.453088 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:31.453097 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:31.453146 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:31.453381 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:31.453381 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:31.453598 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:31.453661 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:31.453663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.454397 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.454398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:31.454903 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:31.454904 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.455222 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.455223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:31.455269 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:31.455779 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:31.455785 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:31.456157 (+ 372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:31.458003 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:31.458003 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:31.458004 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:31.458168 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:31.458368 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:31.458369 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:31.458369 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:31.458412 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:31.458690 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:31.458691 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:31.458740 (+ 49us) client_negotiation.cc:769] Sending connection context
0929 10:50:31.458810 (+ 70us) client_negotiation.cc:240] Negotiation successful
0929 10:50:31.458932 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":84,"threads_started":1}
I20260929 10:50:31.459100 28425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:31.452994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:53433 (local address 127.20.80.254:42259)
0929 10:50:31.453038 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:31.453039 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:31.453158 (+ 119us) server_negotiation.cc:407] Connection header received
0929 10:50:31.453301 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:31.453302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:31.453314 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:31.453368 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:31.453760 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.454300 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.454974 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.455146 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.455210 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:31.456246 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:31.456248 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:31.456248 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:31.456255 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:31.457919 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:31.458238 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:31.458239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:31.458239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:31.458300 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:31.458449 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:31.458450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:31.458450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:31.458578 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:31.458638 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:31.458891 (+ 253us) server_negotiation.cc:299] Negotiation successful
0929 10:50:31.458982 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Sep 29 10:50:31 dist-test-slave-vqch krb5kdc[28329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679031, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 10:50:31 dist-test-slave-vqch krb5kdc[28329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:31 dist-test-slave-vqch krb5kdc[28329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
W20260929 10:50:31.478295 28425 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:31.473254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57644 (local address 127.20.80.254:42259)
0929 10:50:31.473292 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:31.473294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:31.473315 (+ 21us) server_negotiation.cc:407] Connection header received
0929 10:50:31.473341 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:31.473342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:31.473352 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:31.473404 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:31.473994 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.474461 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.475146 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.475315 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.475363 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:31.478183 (+ 2820us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57644: BlockingRecv error: recv got EOF from 127.0.0.1:57644 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
Sep 29 10:50:31 dist-test-slave-vqch krb5kdc[28329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679031, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM
I20260929 10:50:31.485613 28425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:31.479176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57660 (local address 127.20.80.254:42259)
0929 10:50:31.479229 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:31.479231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:31.479328 (+ 97us) server_negotiation.cc:407] Connection header received
0929 10:50:31.479382 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:31.479382 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:31.479390 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:31.479438 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:31.479840 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.480278 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.480899 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.481138 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.481189 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:31.483160 (+ 1971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:31.483161 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:31.483162 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:31.483168 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:31.484549 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:31.484865 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:31.484866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:31.484867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:31.484918 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:31.485234 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:31.485234 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:31.485234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:31.485332 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:31.485411 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:31.485499 (+ 88us) server_negotiation.cc:299] Negotiation successful
0929 10:50:31.485551 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 10:50:31.487126 28373 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57660:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:50:31.488711 28373 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:50:31.495314 28886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:31.491122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:34221 (local address 127.0.0.1:32950)
0929 10:50:31.491595 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:31.491604 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:31.491686 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:31.491952 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:31.491953 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:31.491956 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:31.492034 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:31.492036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.492928 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.492929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:31.494166 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:31.494169 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.495001 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.495002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:31.495082 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:31.495096 (+ 14us) client_negotiation.cc:769] Sending connection context
0929 10:50:31.495168 (+ 72us) client_negotiation.cc:240] Negotiation successful
0929 10:50:31.495216 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"mutex_wait_us":3,"thread_start_us":82,"threads_started":1}
I20260929 10:50:31.495345 28888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:31.491405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32950 (local address 127.20.80.193:34221)
0929 10:50:31.491596 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:31.491601 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:31.491681 (+ 80us) server_negotiation.cc:407] Connection header received
0929 10:50:31.491756 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:31.491759 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:31.491777 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:31.491863 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:31.492117 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.492830 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.494240 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.494913 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.494986 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:31.495060 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:31.495189 (+ 129us) server_negotiation.cc:299] Negotiation successful
0929 10:50:31.495241 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":99,"threads_started":1}
I20260929 10:50:31.495723 28889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:31.491549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55526 (local address 127.20.80.195:41667)
0929 10:50:31.491688 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:31.491692 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:31.491700 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:50:31.491737 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:31.491739 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:31.491760 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:31.492103 (+ 343us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:31.492162 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.493287 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.494483 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.495160 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.495554 (+ 394us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:31.495615 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:31.495635 (+ 20us) server_negotiation.cc:299] Negotiation successful
0929 10:50:31.495663 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":87,"threads_started":1}
I20260929 10:50:31.495501 28885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:31.491122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:41667 (local address 127.0.0.1:55526)
0929 10:50:31.491260 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:31.491298 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:31.491361 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:31.491946 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:31.491949 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:31.491952 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:31.492026 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:31.492028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.493385 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.493386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:31.494409 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:31.494411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.495256 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.495257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:31.495338 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:31.495343 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 10:50:31.495386 (+ 43us) client_negotiation.cc:240] Negotiation successful
0929 10:50:31.495432 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":1,"thread_start_us":58,"threads_started":1}
I20260929 10:50:31.496567 28795 tablet_service.cc:1467] Processing CreateTablet for tablet d81dd255a43d4f108f7e58e30843eafe (DEFAULT_TABLE table=test-table [id=a63f30fcfbe64659a911bd6215984da5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:31.496562 28513 tablet_service.cc:1467] Processing CreateTablet for tablet d81dd255a43d4f108f7e58e30843eafe (DEFAULT_TABLE table=test-table [id=a63f30fcfbe64659a911bd6215984da5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:31.496908 28795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d81dd255a43d4f108f7e58e30843eafe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:31.496915 28513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d81dd255a43d4f108f7e58e30843eafe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:31.499922 28890 tablet_bootstrap.cc:492] T d81dd255a43d4f108f7e58e30843eafe P 97c90aace77b4f56b9f8224b3688aba4: Bootstrap starting.
I20260929 10:50:31.500252 28887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:31.491371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48180 (local address 127.20.80.194:33969)
0929 10:50:31.491542 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:31.491548 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:31.491871 (+ 323us) server_negotiation.cc:407] Connection header received
0929 10:50:31.491914 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:31.491916 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:31.491932 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:31.492000 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:31.492192 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.493277 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.495321 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.496079 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.496172 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:31.496243 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:31.500106 (+ 3863us) server_negotiation.cc:299] Negotiation successful
0929 10:50:31.500160 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":80,"threads_started":1}
I20260929 10:50:31.500331 28891 tablet_bootstrap.cc:492] T d81dd255a43d4f108f7e58e30843eafe P afa84438b9224029840033fd30444c3f: Bootstrap starting.
I20260929 10:50:31.500557 28884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:31.491146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:33969 (local address 127.0.0.1:48180)
0929 10:50:31.491763 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:31.491772 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:31.491821 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:31.492083 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:31.492084 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:31.492086 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:31.492118 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:31.492119 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.494478 (+ 2359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.494479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:31.495235 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:31.495237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.499914 (+ 4677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.499915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:31.499992 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:31.499998 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 10:50:31.500419 (+ 421us) client_negotiation.cc:240] Negotiation successful
0929 10:50:31.500478 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":75,"threads_started":1}
I20260929 10:50:31.500607 28890 tablet_bootstrap.cc:654] T d81dd255a43d4f108f7e58e30843eafe P 97c90aace77b4f56b9f8224b3688aba4: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:31.500960 28890 log.cc:824] T d81dd255a43d4f108f7e58e30843eafe P 97c90aace77b4f56b9f8224b3688aba4: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:31.501122 28891 tablet_bootstrap.cc:654] T d81dd255a43d4f108f7e58e30843eafe P afa84438b9224029840033fd30444c3f: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:31.501345 28654 tablet_service.cc:1467] Processing CreateTablet for tablet d81dd255a43d4f108f7e58e30843eafe (DEFAULT_TABLE table=test-table [id=a63f30fcfbe64659a911bd6215984da5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:31.501457 28891 log.cc:824] T d81dd255a43d4f108f7e58e30843eafe P afa84438b9224029840033fd30444c3f: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:31.501694 28654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d81dd255a43d4f108f7e58e30843eafe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:31.501775 28890 tablet_bootstrap.cc:492] T d81dd255a43d4f108f7e58e30843eafe P 97c90aace77b4f56b9f8224b3688aba4: No bootstrap required, opened a new log
I20260929 10:50:31.501843 28890 ts_tablet_manager.cc:1397] T d81dd255a43d4f108f7e58e30843eafe P 97c90aace77b4f56b9f8224b3688aba4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260929 10:50:31.502147 28891 tablet_bootstrap.cc:492] T d81dd255a43d4f108f7e58e30843eafe P afa84438b9224029840033fd30444c3f: No bootstrap required, opened a new log
I20260929 10:50:31.502228 28891 ts_tablet_manager.cc:1397] T d81dd255a43d4f108f7e58e30843eafe P afa84438b9224029840033fd30444c3f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:50:31.503793 28891 raft_consensus.cc:348] T d81dd255a43d4f108f7e58e30843eafe P afa84438b9224029840033fd30444c3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } } peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } }
I20260929 10:50:31.503945 28891 raft_consensus.cc:374] T d81dd255a43d4f108f7e58e30843eafe P afa84438b9224029840033fd30444c3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:31.503988 28891 raft_consensus.cc:729] T d81dd255a43d4f108f7e58e30843eafe P afa84438b9224029840033fd30444c3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afa84438b9224029840033fd30444c3f, State: Initialized, Role: FOLLOWER
I20260929 10:50:31.503805 28890 raft_consensus.cc:348] T d81dd255a43d4f108f7e58e30843eafe P 97c90aace77b4f56b9f8224b3688aba4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } } peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } }
I20260929 10:50:31.504094 28890 raft_consensus.cc:374] T d81dd255a43d4f108f7e58e30843eafe P 97c90aace77b4f56b9f8224b3688aba4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:31.504160 28890 raft_consensus.cc:729] T d81dd255a43d4f108f7e58e30843eafe P 97c90aace77b4f56b9f8224b3688aba4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97c90aace77b4f56b9f8224b3688aba4, State: Initialized, Role: FOLLOWER
I20260929 10:50:31.504130 28891 consensus_queue.cc:260] T d81dd255a43d4f108f7e58e30843eafe P afa84438b9224029840033fd30444c3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } } peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } }
I20260929 10:50:31.504295 28890 consensus_queue.cc:260] T d81dd255a43d4f108f7e58e30843eafe P 97c90aace77b4f56b9f8224b3688aba4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } } peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } }
I20260929 10:50:31.504465 28891 ts_tablet_manager.cc:1428] T d81dd255a43d4f108f7e58e30843eafe P afa84438b9224029840033fd30444c3f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:50:31.504508 28579 heartbeater.cc:503] Master 127.20.80.254:42259 was elected leader, sending a full tablet report...
I20260929 10:50:31.504585 28890 ts_tablet_manager.cc:1428] T d81dd255a43d4f108f7e58e30843eafe P 97c90aace77b4f56b9f8224b3688aba4: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260929 10:50:31.504711 28861 heartbeater.cc:503] Master 127.20.80.254:42259 was elected leader, sending a full tablet report...
I20260929 10:50:31.505698 28896 tablet_bootstrap.cc:492] T d81dd255a43d4f108f7e58e30843eafe P db5e4a9496764d8d9a2f9bf9fa9df507: Bootstrap starting.
I20260929 10:50:31.506260 28896 tablet_bootstrap.cc:654] T d81dd255a43d4f108f7e58e30843eafe P db5e4a9496764d8d9a2f9bf9fa9df507: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:31.506558 28896 log.cc:824] T d81dd255a43d4f108f7e58e30843eafe P db5e4a9496764d8d9a2f9bf9fa9df507: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:31.507339 28896 tablet_bootstrap.cc:492] T d81dd255a43d4f108f7e58e30843eafe P db5e4a9496764d8d9a2f9bf9fa9df507: No bootstrap required, opened a new log
I20260929 10:50:31.507426 28896 ts_tablet_manager.cc:1397] T d81dd255a43d4f108f7e58e30843eafe P db5e4a9496764d8d9a2f9bf9fa9df507: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:50:31.508824 28896 raft_consensus.cc:348] T d81dd255a43d4f108f7e58e30843eafe P db5e4a9496764d8d9a2f9bf9fa9df507 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } } peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } }
I20260929 10:50:31.508965 28896 raft_consensus.cc:374] T d81dd255a43d4f108f7e58e30843eafe P db5e4a9496764d8d9a2f9bf9fa9df507 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:31.509037 28896 raft_consensus.cc:729] T d81dd255a43d4f108f7e58e30843eafe P db5e4a9496764d8d9a2f9bf9fa9df507 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db5e4a9496764d8d9a2f9bf9fa9df507, State: Initialized, Role: FOLLOWER
I20260929 10:50:31.509205 28896 consensus_queue.cc:260] T d81dd255a43d4f108f7e58e30843eafe P db5e4a9496764d8d9a2f9bf9fa9df507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } } peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } }
I20260929 10:50:31.509465 28896 ts_tablet_manager.cc:1428] T d81dd255a43d4f108f7e58e30843eafe P db5e4a9496764d8d9a2f9bf9fa9df507: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:50:31.509526 28720 heartbeater.cc:503] Master 127.20.80.254:42259 was elected leader, sending a full tablet report...
W20260929 10:50:31.537545 28721 tablet.cc:2369] T d81dd255a43d4f108f7e58e30843eafe P db5e4a9496764d8d9a2f9bf9fa9df507: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:50:31.629860 28580 tablet.cc:2369] T d81dd255a43d4f108f7e58e30843eafe P afa84438b9224029840033fd30444c3f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:50:31.699239 28862 tablet.cc:2369] T d81dd255a43d4f108f7e58e30843eafe P 97c90aace77b4f56b9f8224b3688aba4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:50:31.875536 28895 raft_consensus.cc:482] T d81dd255a43d4f108f7e58e30843eafe P 97c90aace77b4f56b9f8224b3688aba4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:50:31.875669 28895 raft_consensus.cc:504] T d81dd255a43d4f108f7e58e30843eafe P 97c90aace77b4f56b9f8224b3688aba4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } } peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } }
I20260929 10:50:31.876111 28895 leader_election.cc:288] T d81dd255a43d4f108f7e58e30843eafe P 97c90aace77b4f56b9f8224b3688aba4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers afa84438b9224029840033fd30444c3f (127.20.80.193:34221), db5e4a9496764d8d9a2f9bf9fa9df507 (127.20.80.194:33969)
I20260929 10:50:31.879338 28754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:31.876355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:34221 (local address 127.20.80.195:38015)
0929 10:50:31.876423 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:31.876437 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:31.876500 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:31.876707 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:31.876708 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:31.876711 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:31.876828 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:31.876830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.877533 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.877534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:31.878357 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:31.878360 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.879056 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.879057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:31.879152 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:31.879155 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:50:31.879211 (+ 56us) client_negotiation.cc:240] Negotiation successful
0929 10:50:31.879244 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260929 10:50:31.879354 28888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:31.876402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:38015 (local address 127.20.80.193:34221)
0929 10:50:31.876449 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:31.876452 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:31.876486 (+ 34us) server_negotiation.cc:407] Connection header received
0929 10:50:31.876590 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:31.876591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:31.876601 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:31.876675 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:31.876911 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.877457 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.878440 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.878982 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.879037 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:31.879045 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:31.879216 (+ 171us) server_negotiation.cc:299] Negotiation successful
0929 10:50:31.879258 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 10:50:31.879422 28887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:31.876396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:42911 (local address 127.20.80.194:33969)
0929 10:50:31.876464 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:31.876466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:31.876692 (+ 226us) server_negotiation.cc:407] Connection header received
0929 10:50:31.876745 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:31.876746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:31.876755 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:31.876816 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:31.877040 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.877525 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.878616 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.879046 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.879120 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:31.879129 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:31.879309 (+ 180us) server_negotiation.cc:299] Negotiation successful
0929 10:50:31.879352 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 10:50:31.879459 28899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:31.876357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:33969 (local address 127.20.80.195:42911)
0929 10:50:31.876610 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:31.876620 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:31.876686 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:31.876849 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:31.876849 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:31.876851 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:31.876950 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:31.876952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.877605 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.877605 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:31.878533 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:31.878535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.879154 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.879154 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:31.879231 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:31.879235 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:50:31.879290 (+ 55us) client_negotiation.cc:240] Negotiation successful
0929 10:50:31.879327 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":64,"thread_start_us":102,"threads_started":1}
I20260929 10:50:31.879724 28533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d81dd255a43d4f108f7e58e30843eafe" candidate_uuid: "97c90aace77b4f56b9f8224b3688aba4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afa84438b9224029840033fd30444c3f" is_pre_election: true
I20260929 10:50:31.879791 28674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d81dd255a43d4f108f7e58e30843eafe" candidate_uuid: "97c90aace77b4f56b9f8224b3688aba4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" is_pre_election: true
I20260929 10:50:31.879912 28533 raft_consensus.cc:2454] T d81dd255a43d4f108f7e58e30843eafe P afa84438b9224029840033fd30444c3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97c90aace77b4f56b9f8224b3688aba4 in term 0.
I20260929 10:50:31.879931 28674 raft_consensus.cc:2454] T d81dd255a43d4f108f7e58e30843eafe P db5e4a9496764d8d9a2f9bf9fa9df507 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97c90aace77b4f56b9f8224b3688aba4 in term 0.
I20260929 10:50:31.880172 28749 leader_election.cc:302] T d81dd255a43d4f108f7e58e30843eafe P 97c90aace77b4f56b9f8224b3688aba4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97c90aace77b4f56b9f8224b3688aba4, afa84438b9224029840033fd30444c3f; no voters:
I20260929 10:50:31.880342 28895 raft_consensus.cc:2781] T d81dd255a43d4f108f7e58e30843eafe P 97c90aace77b4f56b9f8224b3688aba4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:50:31.880409 28895 raft_consensus.cc:482] T d81dd255a43d4f108f7e58e30843eafe P 97c90aace77b4f56b9f8224b3688aba4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:50:31.880438 28895 raft_consensus.cc:3037] T d81dd255a43d4f108f7e58e30843eafe P 97c90aace77b4f56b9f8224b3688aba4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:31.881440 28895 raft_consensus.cc:504] T d81dd255a43d4f108f7e58e30843eafe P 97c90aace77b4f56b9f8224b3688aba4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } } peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } }
I20260929 10:50:31.881608 28895 leader_election.cc:288] T d81dd255a43d4f108f7e58e30843eafe P 97c90aace77b4f56b9f8224b3688aba4 [CANDIDATE]: Term 1 election: Requested vote from peers afa84438b9224029840033fd30444c3f (127.20.80.193:34221), db5e4a9496764d8d9a2f9bf9fa9df507 (127.20.80.194:33969)
I20260929 10:50:31.881760 28674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d81dd255a43d4f108f7e58e30843eafe" candidate_uuid: "97c90aace77b4f56b9f8224b3688aba4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507"
I20260929 10:50:31.881785 28533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d81dd255a43d4f108f7e58e30843eafe" candidate_uuid: "97c90aace77b4f56b9f8224b3688aba4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afa84438b9224029840033fd30444c3f"
I20260929 10:50:31.881853 28674 raft_consensus.cc:3037] T d81dd255a43d4f108f7e58e30843eafe P db5e4a9496764d8d9a2f9bf9fa9df507 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:31.881876 28533 raft_consensus.cc:3037] T d81dd255a43d4f108f7e58e30843eafe P afa84438b9224029840033fd30444c3f [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:31.882524 28674 raft_consensus.cc:2454] T d81dd255a43d4f108f7e58e30843eafe P db5e4a9496764d8d9a2f9bf9fa9df507 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97c90aace77b4f56b9f8224b3688aba4 in term 1.
I20260929 10:50:31.882584 28533 raft_consensus.cc:2454] T d81dd255a43d4f108f7e58e30843eafe P afa84438b9224029840033fd30444c3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97c90aace77b4f56b9f8224b3688aba4 in term 1.
I20260929 10:50:31.882725 28749 leader_election.cc:302] T d81dd255a43d4f108f7e58e30843eafe P 97c90aace77b4f56b9f8224b3688aba4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97c90aace77b4f56b9f8224b3688aba4, afa84438b9224029840033fd30444c3f; no voters:
I20260929 10:50:31.882839 28895 raft_consensus.cc:2781] T d81dd255a43d4f108f7e58e30843eafe P 97c90aace77b4f56b9f8224b3688aba4 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:31.882969 28895 raft_consensus.cc:686] T d81dd255a43d4f108f7e58e30843eafe P 97c90aace77b4f56b9f8224b3688aba4 [term 1 LEADER]: Becoming Leader. State: Replica: 97c90aace77b4f56b9f8224b3688aba4, State: Running, Role: LEADER
I20260929 10:50:31.883077 28895 consensus_queue.cc:237] T d81dd255a43d4f108f7e58e30843eafe P 97c90aace77b4f56b9f8224b3688aba4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } } peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } }
I20260929 10:50:31.883989 28375 catalog_manager.cc:5495] T d81dd255a43d4f108f7e58e30843eafe P 97c90aace77b4f56b9f8224b3688aba4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 97c90aace77b4f56b9f8224b3688aba4 (127.20.80.195). New cstate: current_term: 1 leader_uuid: "97c90aace77b4f56b9f8224b3688aba4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } health_report { overall_health: UNKNOWN } } }
I20260929 10:50:31.935907 28889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:31.932670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55542 (local address 127.20.80.195:41667)
0929 10:50:31.932730 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:31.932733 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:31.932741 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:50:31.932781 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:31.932781 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:31.932799 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:31.932865 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:50:31.933048 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.933523 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.934745 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:31.935098 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:31.935179 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:31.935453 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:50:31.935610 (+ 157us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:50:31.935701 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:31.935758 (+ 57us) server_negotiation.cc:299] Negotiation successful
0929 10:50:31.935786 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260929 10:50:31.939476 28674 raft_consensus.cc:1261] T d81dd255a43d4f108f7e58e30843eafe P db5e4a9496764d8d9a2f9bf9fa9df507 [term 1 FOLLOWER]: Refusing update from remote peer 97c90aace77b4f56b9f8224b3688aba4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:31.939514 28533 raft_consensus.cc:1261] T d81dd255a43d4f108f7e58e30843eafe P afa84438b9224029840033fd30444c3f [term 1 FOLLOWER]: Refusing update from remote peer 97c90aace77b4f56b9f8224b3688aba4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:31.939910 28900 consensus_queue.cc:1035] T d81dd255a43d4f108f7e58e30843eafe P 97c90aace77b4f56b9f8224b3688aba4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:50:31.941076 28900 consensus_queue.cc:1035] T d81dd255a43d4f108f7e58e30843eafe P 97c90aace77b4f56b9f8224b3688aba4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:50:31.942373 28903 mvcc.cc:205] Tried to move back new op lower bound from 7334621314815619072 to 7334621314593644544. Current Snapshot: MvccSnapshot[applied={T|T < 7334621314815619072}]
I20260929 10:50:31.942477 28904 mvcc.cc:205] Tried to move back new op lower bound from 7334621314815619072 to 7334621314593644544. Current Snapshot: MvccSnapshot[applied={T|T < 7334621314815619072}]
I20260929 10:50:31.943701 28906 mvcc.cc:205] Tried to move back new op lower bound from 7334621314815619072 to 7334621314593644544. Current Snapshot: MvccSnapshot[applied={T|T < 7334621314815619072}]
I20260929 10:50:31.945175 28375 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:50:31.945268 28375 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:50:31.946026 28375 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c04d4b30fd884a0f83bf748dc15c3a26: Sending DeleteTablet for 3 replicas of tablet d81dd255a43d4f108f7e58e30843eafe
I20260929 10:50:31.946360 28654 tablet_service.cc:1514] Processing DeleteTablet for tablet d81dd255a43d4f108f7e58e30843eafe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:31 UTC) from {username='oryx', principal='oryx/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:48180
I20260929 10:50:31.946355 28795 tablet_service.cc:1514] Processing DeleteTablet for tablet d81dd255a43d4f108f7e58e30843eafe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:31 UTC) from {username='oryx', principal='oryx/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:55526
I20260929 10:50:31.946413 28513 tablet_service.cc:1514] Processing DeleteTablet for tablet d81dd255a43d4f108f7e58e30843eafe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:31 UTC) from {username='oryx', principal='oryx/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:32950
I20260929 10:50:31.946592 28914 tablet_replica.cc:330] T d81dd255a43d4f108f7e58e30843eafe P afa84438b9224029840033fd30444c3f: stopping tablet replica
I20260929 10:50:31.946691 28914 raft_consensus.cc:2229] T d81dd255a43d4f108f7e58e30843eafe P afa84438b9224029840033fd30444c3f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:50:31.946709 28375 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57660:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 10:50:31.946802 28914 raft_consensus.cc:2258] T d81dd255a43d4f108f7e58e30843eafe P afa84438b9224029840033fd30444c3f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260929 10:50:31.946866 28375 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:50:31.947361 28914 ts_tablet_manager.cc:1905] T d81dd255a43d4f108f7e58e30843eafe P afa84438b9224029840033fd30444c3f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:31.948074 28513 tablet_service.cc:1467] Processing CreateTablet for tablet fe19778390e84b8885646db70bd72991 (DEFAULT_TABLE table=test-table [id=fa2153933bc34b4d9a3fa98e9aac237a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:31.948076 28654 tablet_service.cc:1467] Processing CreateTablet for tablet fe19778390e84b8885646db70bd72991 (DEFAULT_TABLE table=test-table [id=fa2153933bc34b4d9a3fa98e9aac237a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:31.948189 28513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe19778390e84b8885646db70bd72991. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:31.948189 28654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe19778390e84b8885646db70bd72991. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:31.949374 28914 ts_tablet_manager.cc:1918] T d81dd255a43d4f108f7e58e30843eafe P afa84438b9224029840033fd30444c3f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:50:31.949440 28914 log.cc:1196] T d81dd255a43d4f108f7e58e30843eafe P afa84438b9224029840033fd30444c3f: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-0/wal/wals/d81dd255a43d4f108f7e58e30843eafe
I20260929 10:50:31.949645 28891 tablet_bootstrap.cc:492] T fe19778390e84b8885646db70bd72991 P afa84438b9224029840033fd30444c3f: Bootstrap starting.
I20260929 10:50:31.949697 28914 ts_tablet_manager.cc:1939] T d81dd255a43d4f108f7e58e30843eafe P afa84438b9224029840033fd30444c3f: Deleting consensus metadata
I20260929 10:50:31.949963 28795 tablet_service.cc:1467] Processing CreateTablet for tablet fe19778390e84b8885646db70bd72991 (DEFAULT_TABLE table=test-table [id=fa2153933bc34b4d9a3fa98e9aac237a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:31.950057 28891 tablet_bootstrap.cc:654] T fe19778390e84b8885646db70bd72991 P afa84438b9224029840033fd30444c3f: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:31.950068 28795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe19778390e84b8885646db70bd72991. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:31.950080 28361 catalog_manager.cc:4841] TS afa84438b9224029840033fd30444c3f (127.20.80.193:34221): tablet d81dd255a43d4f108f7e58e30843eafe (table test-table [id=a63f30fcfbe64659a911bd6215984da5]) successfully deleted
I20260929 10:50:31.950464 28896 tablet_bootstrap.cc:492] T fe19778390e84b8885646db70bd72991 P db5e4a9496764d8d9a2f9bf9fa9df507: Bootstrap starting.
I20260929 10:50:31.950647 28891 tablet_bootstrap.cc:492] T fe19778390e84b8885646db70bd72991 P afa84438b9224029840033fd30444c3f: No bootstrap required, opened a new log
I20260929 10:50:31.950685 28891 ts_tablet_manager.cc:1397] T fe19778390e84b8885646db70bd72991 P afa84438b9224029840033fd30444c3f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:31.950824 28896 tablet_bootstrap.cc:654] T fe19778390e84b8885646db70bd72991 P db5e4a9496764d8d9a2f9bf9fa9df507: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:31.950827 28891 raft_consensus.cc:348] T fe19778390e84b8885646db70bd72991 P afa84438b9224029840033fd30444c3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } }
I20260929 10:50:31.950902 28891 raft_consensus.cc:374] T fe19778390e84b8885646db70bd72991 P afa84438b9224029840033fd30444c3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:31.950923 28891 raft_consensus.cc:729] T fe19778390e84b8885646db70bd72991 P afa84438b9224029840033fd30444c3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afa84438b9224029840033fd30444c3f, State: Initialized, Role: FOLLOWER
I20260929 10:50:31.950968 28891 consensus_queue.cc:260] T fe19778390e84b8885646db70bd72991 P afa84438b9224029840033fd30444c3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } }
I20260929 10:50:31.951283 28891 ts_tablet_manager.cc:1428] T fe19778390e84b8885646db70bd72991 P afa84438b9224029840033fd30444c3f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260929 10:50:31.951552 28890 tablet_bootstrap.cc:492] T fe19778390e84b8885646db70bd72991 P 97c90aace77b4f56b9f8224b3688aba4: Bootstrap starting.
I20260929 10:50:31.951875 28896 tablet_bootstrap.cc:492] T fe19778390e84b8885646db70bd72991 P db5e4a9496764d8d9a2f9bf9fa9df507: No bootstrap required, opened a new log
I20260929 10:50:31.951917 28896 ts_tablet_manager.cc:1397] T fe19778390e84b8885646db70bd72991 P db5e4a9496764d8d9a2f9bf9fa9df507: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:31.951962 28890 tablet_bootstrap.cc:654] T fe19778390e84b8885646db70bd72991 P 97c90aace77b4f56b9f8224b3688aba4: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:31.952060 28896 raft_consensus.cc:348] T fe19778390e84b8885646db70bd72991 P db5e4a9496764d8d9a2f9bf9fa9df507 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } }
I20260929 10:50:31.952144 28896 raft_consensus.cc:374] T fe19778390e84b8885646db70bd72991 P db5e4a9496764d8d9a2f9bf9fa9df507 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:31.952175 28896 raft_consensus.cc:729] T fe19778390e84b8885646db70bd72991 P db5e4a9496764d8d9a2f9bf9fa9df507 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db5e4a9496764d8d9a2f9bf9fa9df507, State: Initialized, Role: FOLLOWER
I20260929 10:50:31.952214 28896 consensus_queue.cc:260] T fe19778390e84b8885646db70bd72991 P db5e4a9496764d8d9a2f9bf9fa9df507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } }
I20260929 10:50:31.952320 28896 ts_tablet_manager.cc:1428] T fe19778390e84b8885646db70bd72991 P db5e4a9496764d8d9a2f9bf9fa9df507: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:31.952559 28890 tablet_bootstrap.cc:492] T fe19778390e84b8885646db70bd72991 P 97c90aace77b4f56b9f8224b3688aba4: No bootstrap required, opened a new log
I20260929 10:50:31.952603 28890 ts_tablet_manager.cc:1397] T fe19778390e84b8885646db70bd72991 P 97c90aace77b4f56b9f8224b3688aba4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 10:50:31.952750 28890 raft_consensus.cc:348] T fe19778390e84b8885646db70bd72991 P 97c90aace77b4f56b9f8224b3688aba4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } }
I20260929 10:50:31.952821 28890 raft_consensus.cc:374] T fe19778390e84b8885646db70bd72991 P 97c90aace77b4f56b9f8224b3688aba4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:31.952840 28890 raft_consensus.cc:729] T fe19778390e84b8885646db70bd72991 P 97c90aace77b4f56b9f8224b3688aba4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97c90aace77b4f56b9f8224b3688aba4, State: Initialized, Role: FOLLOWER
I20260929 10:50:31.952877 28890 consensus_queue.cc:260] T fe19778390e84b8885646db70bd72991 P 97c90aace77b4f56b9f8224b3688aba4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } }
I20260929 10:50:31.952960 28890 ts_tablet_manager.cc:1428] T fe19778390e84b8885646db70bd72991 P 97c90aace77b4f56b9f8224b3688aba4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:31.957105 28913 tablet_replica.cc:330] T d81dd255a43d4f108f7e58e30843eafe P 97c90aace77b4f56b9f8224b3688aba4: stopping tablet replica
I20260929 10:50:31.957219 28913 raft_consensus.cc:2229] T d81dd255a43d4f108f7e58e30843eafe P 97c90aace77b4f56b9f8224b3688aba4 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:50:31.957320 28913 raft_consensus.cc:2258] T d81dd255a43d4f108f7e58e30843eafe P 97c90aace77b4f56b9f8224b3688aba4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:31.957464 28912 tablet_replica.cc:330] T d81dd255a43d4f108f7e58e30843eafe P db5e4a9496764d8d9a2f9bf9fa9df507: stopping tablet replica
I20260929 10:50:31.957564 28912 raft_consensus.cc:2229] T d81dd255a43d4f108f7e58e30843eafe P db5e4a9496764d8d9a2f9bf9fa9df507 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:50:31.957680 28912 raft_consensus.cc:2258] T d81dd255a43d4f108f7e58e30843eafe P db5e4a9496764d8d9a2f9bf9fa9df507 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:31.957813 28913 ts_tablet_manager.cc:1905] T d81dd255a43d4f108f7e58e30843eafe P 97c90aace77b4f56b9f8224b3688aba4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:31.958108 28912 ts_tablet_manager.cc:1905] T d81dd255a43d4f108f7e58e30843eafe P db5e4a9496764d8d9a2f9bf9fa9df507: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:31.959481 28913 ts_tablet_manager.cc:1918] T d81dd255a43d4f108f7e58e30843eafe P 97c90aace77b4f56b9f8224b3688aba4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:50:31.959568 28913 log.cc:1196] T d81dd255a43d4f108f7e58e30843eafe P 97c90aace77b4f56b9f8224b3688aba4: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-2/wal/wals/d81dd255a43d4f108f7e58e30843eafe
I20260929 10:50:31.959827 28913 ts_tablet_manager.cc:1939] T d81dd255a43d4f108f7e58e30843eafe P 97c90aace77b4f56b9f8224b3688aba4: Deleting consensus metadata
I20260929 10:50:31.960210 28360 catalog_manager.cc:4841] TS 97c90aace77b4f56b9f8224b3688aba4 (127.20.80.195:41667): tablet d81dd255a43d4f108f7e58e30843eafe (table test-table [id=a63f30fcfbe64659a911bd6215984da5]) successfully deleted
I20260929 10:50:31.963649 28912 ts_tablet_manager.cc:1918] T d81dd255a43d4f108f7e58e30843eafe P db5e4a9496764d8d9a2f9bf9fa9df507: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:50:31.963723 28912 log.cc:1196] T d81dd255a43d4f108f7e58e30843eafe P db5e4a9496764d8d9a2f9bf9fa9df507: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-1/wal/wals/d81dd255a43d4f108f7e58e30843eafe
I20260929 10:50:31.963999 28912 ts_tablet_manager.cc:1939] T d81dd255a43d4f108f7e58e30843eafe P db5e4a9496764d8d9a2f9bf9fa9df507: Deleting consensus metadata
I20260929 10:50:31.964409 28359 catalog_manager.cc:4841] TS db5e4a9496764d8d9a2f9bf9fa9df507 (127.20.80.194:33969): tablet d81dd255a43d4f108f7e58e30843eafe (table test-table [id=a63f30fcfbe64659a911bd6215984da5]) successfully deleted
I20260929 10:50:31.982302 28900 raft_consensus.cc:482] T fe19778390e84b8885646db70bd72991 P 97c90aace77b4f56b9f8224b3688aba4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:50:31.982410 28900 raft_consensus.cc:504] T fe19778390e84b8885646db70bd72991 P 97c90aace77b4f56b9f8224b3688aba4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } }
I20260929 10:50:31.982594 28900 leader_election.cc:288] T fe19778390e84b8885646db70bd72991 P 97c90aace77b4f56b9f8224b3688aba4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers afa84438b9224029840033fd30444c3f (127.20.80.193:34221), db5e4a9496764d8d9a2f9bf9fa9df507 (127.20.80.194:33969)
I20260929 10:50:31.982838 28533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe19778390e84b8885646db70bd72991" candidate_uuid: "97c90aace77b4f56b9f8224b3688aba4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afa84438b9224029840033fd30444c3f" is_pre_election: true
I20260929 10:50:31.982831 28674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe19778390e84b8885646db70bd72991" candidate_uuid: "97c90aace77b4f56b9f8224b3688aba4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" is_pre_election: true
I20260929 10:50:31.982955 28533 raft_consensus.cc:2454] T fe19778390e84b8885646db70bd72991 P afa84438b9224029840033fd30444c3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97c90aace77b4f56b9f8224b3688aba4 in term 0.
I20260929 10:50:31.982950 28674 raft_consensus.cc:2454] T fe19778390e84b8885646db70bd72991 P db5e4a9496764d8d9a2f9bf9fa9df507 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97c90aace77b4f56b9f8224b3688aba4 in term 0.
I20260929 10:50:31.983106 28749 leader_election.cc:302] T fe19778390e84b8885646db70bd72991 P 97c90aace77b4f56b9f8224b3688aba4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97c90aace77b4f56b9f8224b3688aba4, afa84438b9224029840033fd30444c3f; no voters:
I20260929 10:50:31.983192 28900 raft_consensus.cc:2781] T fe19778390e84b8885646db70bd72991 P 97c90aace77b4f56b9f8224b3688aba4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:50:31.983220 28900 raft_consensus.cc:482] T fe19778390e84b8885646db70bd72991 P 97c90aace77b4f56b9f8224b3688aba4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:50:31.983240 28900 raft_consensus.cc:3037] T fe19778390e84b8885646db70bd72991 P 97c90aace77b4f56b9f8224b3688aba4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:31.984172 28900 raft_consensus.cc:504] T fe19778390e84b8885646db70bd72991 P 97c90aace77b4f56b9f8224b3688aba4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } }
I20260929 10:50:31.984350 28900 leader_election.cc:288] T fe19778390e84b8885646db70bd72991 P 97c90aace77b4f56b9f8224b3688aba4 [CANDIDATE]: Term 1 election: Requested vote from peers afa84438b9224029840033fd30444c3f (127.20.80.193:34221), db5e4a9496764d8d9a2f9bf9fa9df507 (127.20.80.194:33969)
I20260929 10:50:31.984525 28674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe19778390e84b8885646db70bd72991" candidate_uuid: "97c90aace77b4f56b9f8224b3688aba4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507"
I20260929 10:50:31.984560 28533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe19778390e84b8885646db70bd72991" candidate_uuid: "97c90aace77b4f56b9f8224b3688aba4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afa84438b9224029840033fd30444c3f"
I20260929 10:50:31.984622 28674 raft_consensus.cc:3037] T fe19778390e84b8885646db70bd72991 P db5e4a9496764d8d9a2f9bf9fa9df507 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:31.984627 28533 raft_consensus.cc:3037] T fe19778390e84b8885646db70bd72991 P afa84438b9224029840033fd30444c3f [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:31.985437 28674 raft_consensus.cc:2454] T fe19778390e84b8885646db70bd72991 P db5e4a9496764d8d9a2f9bf9fa9df507 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97c90aace77b4f56b9f8224b3688aba4 in term 1.
I20260929 10:50:31.985437 28533 raft_consensus.cc:2454] T fe19778390e84b8885646db70bd72991 P afa84438b9224029840033fd30444c3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97c90aace77b4f56b9f8224b3688aba4 in term 1.
I20260929 10:50:31.985610 28747 leader_election.cc:302] T fe19778390e84b8885646db70bd72991 P 97c90aace77b4f56b9f8224b3688aba4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97c90aace77b4f56b9f8224b3688aba4, db5e4a9496764d8d9a2f9bf9fa9df507; no voters:
I20260929 10:50:31.985708 28900 raft_consensus.cc:2781] T fe19778390e84b8885646db70bd72991 P 97c90aace77b4f56b9f8224b3688aba4 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:31.985749 28900 raft_consensus.cc:686] T fe19778390e84b8885646db70bd72991 P 97c90aace77b4f56b9f8224b3688aba4 [term 1 LEADER]: Becoming Leader. State: Replica: 97c90aace77b4f56b9f8224b3688aba4, State: Running, Role: LEADER
I20260929 10:50:31.985793 28900 consensus_queue.cc:237] T fe19778390e84b8885646db70bd72991 P 97c90aace77b4f56b9f8224b3688aba4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } }
I20260929 10:50:31.986343 28375 catalog_manager.cc:5495] T fe19778390e84b8885646db70bd72991 P 97c90aace77b4f56b9f8224b3688aba4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 97c90aace77b4f56b9f8224b3688aba4 (127.20.80.195). New cstate: current_term: 1 leader_uuid: "97c90aace77b4f56b9f8224b3688aba4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } health_report { overall_health: HEALTHY } } }
Sep 29 10:50:32 dist-test-slave-vqch krb5kdc[28329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679030, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.254@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM
I20260929 10:50:32.021771 28924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:32.013773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:42259 (local address 127.0.0.1:57672)
0929 10:50:32.013883 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:32.013895 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:32.013962 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:32.014145 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:32.014146 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:32.014443 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:32.014495 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:32.014497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:32.015461 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:32.015462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:32.016030 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:32.016031 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:32.016534 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:32.016536 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:32.016584 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:32.017242 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:32.017246 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:32.018616 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:32.020483 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:32.020491 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:32.020493 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:32.020723 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:32.020960 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:32.020961 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:32.020961 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:32.021102 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:32.021490 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:32.021497 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:32.021541 (+ 44us) client_negotiation.cc:769] Sending connection context
0929 10:50:32.021604 (+ 63us) client_negotiation.cc:240] Negotiation successful
0929 10:50:32.021694 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":64,"threads_started":1}
I20260929 10:50:32.021885 28925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:32.013803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57672 (local address 127.20.80.254:42259)
0929 10:50:32.013918 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:32.013920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:32.013955 (+ 35us) server_negotiation.cc:407] Connection header received
0929 10:50:32.014030 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:32.014031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:32.014043 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:32.014127 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:32.014590 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:32.015363 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:32.016132 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:32.016435 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:32.016520 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:32.018714 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:32.018716 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:32.018716 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:32.018724 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:32.020385 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:32.020816 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:32.020817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:32.020818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:32.020885 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:32.021158 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:32.021159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:32.021159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:32.021367 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:32.021458 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:32.021659 (+ 201us) server_negotiation.cc:299] Negotiation successful
0929 10:50:32.021759 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":59,"threads_started":1}
I20260929 10:50:32.024083 28375 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:57672:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 10:50:32.024330 28375 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:50:32.025847 28795 tablet_service.cc:1467] Processing CreateTablet for tablet 9d9f19b02ce64df5af2e9901b7418dce (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bc1fa0353e4c48e8921650cb4842f1a2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 10:50:32.025893 28654 tablet_service.cc:1467] Processing CreateTablet for tablet 9d9f19b02ce64df5af2e9901b7418dce (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bc1fa0353e4c48e8921650cb4842f1a2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 10:50:32.025894 28513 tablet_service.cc:1467] Processing CreateTablet for tablet 9d9f19b02ce64df5af2e9901b7418dce (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bc1fa0353e4c48e8921650cb4842f1a2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 10:50:32.026008 28795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d9f19b02ce64df5af2e9901b7418dce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:32.026036 28513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d9f19b02ce64df5af2e9901b7418dce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:32.026036 28654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d9f19b02ce64df5af2e9901b7418dce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:32.027541 28896 tablet_bootstrap.cc:492] T 9d9f19b02ce64df5af2e9901b7418dce P db5e4a9496764d8d9a2f9bf9fa9df507: Bootstrap starting.
I20260929 10:50:32.027541 28891 tablet_bootstrap.cc:492] T 9d9f19b02ce64df5af2e9901b7418dce P afa84438b9224029840033fd30444c3f: Bootstrap starting.
I20260929 10:50:32.027546 28890 tablet_bootstrap.cc:492] T 9d9f19b02ce64df5af2e9901b7418dce P 97c90aace77b4f56b9f8224b3688aba4: Bootstrap starting.
I20260929 10:50:32.027945 28890 tablet_bootstrap.cc:654] T 9d9f19b02ce64df5af2e9901b7418dce P 97c90aace77b4f56b9f8224b3688aba4: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:32.027959 28891 tablet_bootstrap.cc:654] T 9d9f19b02ce64df5af2e9901b7418dce P afa84438b9224029840033fd30444c3f: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:32.027959 28896 tablet_bootstrap.cc:654] T 9d9f19b02ce64df5af2e9901b7418dce P db5e4a9496764d8d9a2f9bf9fa9df507: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:32.028899 28896 tablet_bootstrap.cc:492] T 9d9f19b02ce64df5af2e9901b7418dce P db5e4a9496764d8d9a2f9bf9fa9df507: No bootstrap required, opened a new log
I20260929 10:50:32.028944 28890 tablet_bootstrap.cc:492] T 9d9f19b02ce64df5af2e9901b7418dce P 97c90aace77b4f56b9f8224b3688aba4: No bootstrap required, opened a new log
I20260929 10:50:32.028971 28896 ts_tablet_manager.cc:1397] T 9d9f19b02ce64df5af2e9901b7418dce P db5e4a9496764d8d9a2f9bf9fa9df507: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:32.029014 28890 ts_tablet_manager.cc:1397] T 9d9f19b02ce64df5af2e9901b7418dce P 97c90aace77b4f56b9f8224b3688aba4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 10:50:32.029170 28896 raft_consensus.cc:348] T 9d9f19b02ce64df5af2e9901b7418dce P db5e4a9496764d8d9a2f9bf9fa9df507 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } }
I20260929 10:50:32.029171 28890 raft_consensus.cc:348] T 9d9f19b02ce64df5af2e9901b7418dce P 97c90aace77b4f56b9f8224b3688aba4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } }
I20260929 10:50:32.029258 28890 raft_consensus.cc:374] T 9d9f19b02ce64df5af2e9901b7418dce P 97c90aace77b4f56b9f8224b3688aba4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:32.029258 28896 raft_consensus.cc:374] T 9d9f19b02ce64df5af2e9901b7418dce P db5e4a9496764d8d9a2f9bf9fa9df507 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:32.029300 28896 raft_consensus.cc:729] T 9d9f19b02ce64df5af2e9901b7418dce P db5e4a9496764d8d9a2f9bf9fa9df507 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db5e4a9496764d8d9a2f9bf9fa9df507, State: Initialized, Role: FOLLOWER
I20260929 10:50:32.029301 28890 raft_consensus.cc:729] T 9d9f19b02ce64df5af2e9901b7418dce P 97c90aace77b4f56b9f8224b3688aba4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97c90aace77b4f56b9f8224b3688aba4, State: Initialized, Role: FOLLOWER
I20260929 10:50:32.029345 28896 consensus_queue.cc:260] T 9d9f19b02ce64df5af2e9901b7418dce P db5e4a9496764d8d9a2f9bf9fa9df507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } }
I20260929 10:50:32.029347 28890 consensus_queue.cc:260] T 9d9f19b02ce64df5af2e9901b7418dce P 97c90aace77b4f56b9f8224b3688aba4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } }
I20260929 10:50:32.029446 28896 ts_tablet_manager.cc:1428] T 9d9f19b02ce64df5af2e9901b7418dce P db5e4a9496764d8d9a2f9bf9fa9df507: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:32.029454 28890 ts_tablet_manager.cc:1428] T 9d9f19b02ce64df5af2e9901b7418dce P 97c90aace77b4f56b9f8224b3688aba4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:32.029523 28898 tablet_replica.cc:439] T 9d9f19b02ce64df5af2e9901b7418dce P db5e4a9496764d8d9a2f9bf9fa9df507: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } }
I20260929 10:50:32.029548 28900 tablet_replica.cc:439] T 9d9f19b02ce64df5af2e9901b7418dce P 97c90aace77b4f56b9f8224b3688aba4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } }
I20260929 10:50:32.029611 28900 tablet_replica.cc:442] T 9d9f19b02ce64df5af2e9901b7418dce P 97c90aace77b4f56b9f8224b3688aba4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:50:32.029608 28898 tablet_replica.cc:442] T 9d9f19b02ce64df5af2e9901b7418dce P db5e4a9496764d8d9a2f9bf9fa9df507: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:50:32.029846 28891 tablet_bootstrap.cc:492] T 9d9f19b02ce64df5af2e9901b7418dce P afa84438b9224029840033fd30444c3f: No bootstrap required, opened a new log
I20260929 10:50:32.029887 28891 ts_tablet_manager.cc:1397] T 9d9f19b02ce64df5af2e9901b7418dce P afa84438b9224029840033fd30444c3f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:50:32.030020 28891 raft_consensus.cc:348] T 9d9f19b02ce64df5af2e9901b7418dce P afa84438b9224029840033fd30444c3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } }
I20260929 10:50:32.030092 28891 raft_consensus.cc:374] T 9d9f19b02ce64df5af2e9901b7418dce P afa84438b9224029840033fd30444c3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:32.030112 28891 raft_consensus.cc:729] T 9d9f19b02ce64df5af2e9901b7418dce P afa84438b9224029840033fd30444c3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afa84438b9224029840033fd30444c3f, State: Initialized, Role: FOLLOWER
I20260929 10:50:32.030158 28891 consensus_queue.cc:260] T 9d9f19b02ce64df5af2e9901b7418dce P afa84438b9224029840033fd30444c3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } }
I20260929 10:50:32.030246 28891 ts_tablet_manager.cc:1428] T 9d9f19b02ce64df5af2e9901b7418dce P afa84438b9224029840033fd30444c3f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:32.030371 28894 tablet_replica.cc:439] T 9d9f19b02ce64df5af2e9901b7418dce P afa84438b9224029840033fd30444c3f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } }
I20260929 10:50:32.030444 28894 tablet_replica.cc:442] T 9d9f19b02ce64df5af2e9901b7418dce P afa84438b9224029840033fd30444c3f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:50:32.098506 28894 raft_consensus.cc:482] T 9d9f19b02ce64df5af2e9901b7418dce P afa84438b9224029840033fd30444c3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:50:32.098646 28894 raft_consensus.cc:504] T 9d9f19b02ce64df5af2e9901b7418dce P afa84438b9224029840033fd30444c3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } }
I20260929 10:50:32.099076 28894 leader_election.cc:288] T 9d9f19b02ce64df5af2e9901b7418dce P afa84438b9224029840033fd30444c3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97c90aace77b4f56b9f8224b3688aba4 (127.20.80.195:41667), db5e4a9496764d8d9a2f9bf9fa9df507 (127.20.80.194:33969)
I20260929 10:50:32.103305 28889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:32.099443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:37355 (local address 127.20.80.195:41667)
0929 10:50:32.099574 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:32.099576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:32.099591 (+ 15us) server_negotiation.cc:407] Connection header received
0929 10:50:32.099689 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:32.099691 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:32.099705 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:32.099738 (+ 33us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:32.100008 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:32.100771 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:32.102332 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:32.102891 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:32.102983 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:32.102994 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:32.103185 (+ 191us) server_negotiation.cc:299] Negotiation successful
0929 10:50:32.103237 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260929 10:50:32.103305 28887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:32.099460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:51273 (local address 127.20.80.194:33969)
0929 10:50:32.099521 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:32.099523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:32.099735 (+ 212us) server_negotiation.cc:407] Connection header received
0929 10:50:32.099848 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:32.099849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:32.099859 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:32.099943 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:32.100176 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:32.100969 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:32.102332 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:32.102931 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:32.102996 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:32.103004 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:32.103188 (+ 184us) server_negotiation.cc:299] Negotiation successful
0929 10:50:32.103235 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 10:50:32.103313 28930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:32.099261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:41667 (local address 127.20.80.193:37355)
0929 10:50:32.099512 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:32.099524 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:32.099587 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:32.099823 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:32.099824 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:32.099826 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:32.099909 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:32.099911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:32.100862 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:32.100862 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:32.102227 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:32.102229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:32.102999 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:32.103001 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:32.103084 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:32.103089 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 10:50:32.103149 (+ 60us) client_negotiation.cc:240] Negotiation successful
0929 10:50:32.103199 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":28,"thread_start_us":118,"threads_started":1}
I20260929 10:50:32.103317 28929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:32.099283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:33969 (local address 127.20.80.193:51273)
0929 10:50:32.099664 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:32.099676 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:32.099751 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:32.099958 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:32.099959 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:32.099961 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:32.100065 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:32.100067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:32.101090 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:32.101091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:32.102220 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:32.102223 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:32.103008 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:32.103009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:32.103084 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:32.103089 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 10:50:32.103149 (+ 60us) client_negotiation.cc:240] Negotiation successful
0929 10:50:32.103200 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":97,"threads_started":1}
I20260929 10:50:32.103606 28815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d9f19b02ce64df5af2e9901b7418dce" candidate_uuid: "afa84438b9224029840033fd30444c3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97c90aace77b4f56b9f8224b3688aba4" is_pre_election: true
I20260929 10:50:32.103699 28674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d9f19b02ce64df5af2e9901b7418dce" candidate_uuid: "afa84438b9224029840033fd30444c3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" is_pre_election: true
I20260929 10:50:32.103775 28674 raft_consensus.cc:2454] T 9d9f19b02ce64df5af2e9901b7418dce P db5e4a9496764d8d9a2f9bf9fa9df507 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afa84438b9224029840033fd30444c3f in term 0.
I20260929 10:50:32.103790 28815 raft_consensus.cc:2454] T 9d9f19b02ce64df5af2e9901b7418dce P 97c90aace77b4f56b9f8224b3688aba4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afa84438b9224029840033fd30444c3f in term 0.
I20260929 10:50:32.103965 28465 leader_election.cc:302] T 9d9f19b02ce64df5af2e9901b7418dce P afa84438b9224029840033fd30444c3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: afa84438b9224029840033fd30444c3f, db5e4a9496764d8d9a2f9bf9fa9df507; no voters:
I20260929 10:50:32.104117 28894 raft_consensus.cc:2781] T 9d9f19b02ce64df5af2e9901b7418dce P afa84438b9224029840033fd30444c3f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:50:32.104156 28894 raft_consensus.cc:482] T 9d9f19b02ce64df5af2e9901b7418dce P afa84438b9224029840033fd30444c3f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:50:32.104179 28894 raft_consensus.cc:3037] T 9d9f19b02ce64df5af2e9901b7418dce P afa84438b9224029840033fd30444c3f [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:32.104908 28894 raft_consensus.cc:504] T 9d9f19b02ce64df5af2e9901b7418dce P afa84438b9224029840033fd30444c3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } }
I20260929 10:50:32.105077 28894 leader_election.cc:288] T 9d9f19b02ce64df5af2e9901b7418dce P afa84438b9224029840033fd30444c3f [CANDIDATE]: Term 1 election: Requested vote from peers 97c90aace77b4f56b9f8224b3688aba4 (127.20.80.195:41667), db5e4a9496764d8d9a2f9bf9fa9df507 (127.20.80.194:33969)
I20260929 10:50:32.105257 28674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d9f19b02ce64df5af2e9901b7418dce" candidate_uuid: "afa84438b9224029840033fd30444c3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507"
I20260929 10:50:32.105258 28815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d9f19b02ce64df5af2e9901b7418dce" candidate_uuid: "afa84438b9224029840033fd30444c3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97c90aace77b4f56b9f8224b3688aba4"
I20260929 10:50:32.105331 28674 raft_consensus.cc:3037] T 9d9f19b02ce64df5af2e9901b7418dce P db5e4a9496764d8d9a2f9bf9fa9df507 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:32.105331 28815 raft_consensus.cc:3037] T 9d9f19b02ce64df5af2e9901b7418dce P 97c90aace77b4f56b9f8224b3688aba4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:32.106086 28674 raft_consensus.cc:2454] T 9d9f19b02ce64df5af2e9901b7418dce P db5e4a9496764d8d9a2f9bf9fa9df507 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afa84438b9224029840033fd30444c3f in term 1.
I20260929 10:50:32.106094 28815 raft_consensus.cc:2454] T 9d9f19b02ce64df5af2e9901b7418dce P 97c90aace77b4f56b9f8224b3688aba4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afa84438b9224029840033fd30444c3f in term 1.
I20260929 10:50:32.106240 28465 leader_election.cc:302] T 9d9f19b02ce64df5af2e9901b7418dce P afa84438b9224029840033fd30444c3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: afa84438b9224029840033fd30444c3f, db5e4a9496764d8d9a2f9bf9fa9df507; no voters:
I20260929 10:50:32.106335 28894 raft_consensus.cc:2781] T 9d9f19b02ce64df5af2e9901b7418dce P afa84438b9224029840033fd30444c3f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:32.106479 28894 raft_consensus.cc:686] T 9d9f19b02ce64df5af2e9901b7418dce P afa84438b9224029840033fd30444c3f [term 1 LEADER]: Becoming Leader. State: Replica: afa84438b9224029840033fd30444c3f, State: Running, Role: LEADER
I20260929 10:50:32.106534 28894 consensus_queue.cc:237] T 9d9f19b02ce64df5af2e9901b7418dce P afa84438b9224029840033fd30444c3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } }
I20260929 10:50:32.106918 28931 tablet_replica.cc:439] T 9d9f19b02ce64df5af2e9901b7418dce P afa84438b9224029840033fd30444c3f: TxnStatusTablet state changed. Reason: New leader afa84438b9224029840033fd30444c3f. Latest consensus state: current_term: 1 leader_uuid: "afa84438b9224029840033fd30444c3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } }
I20260929 10:50:32.107019 28931 tablet_replica.cc:442] T 9d9f19b02ce64df5af2e9901b7418dce P afa84438b9224029840033fd30444c3f: This TxnStatusTablet replica's current role is: LEADER
I20260929 10:50:32.107193 28933 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 10:50:32.107322 28933 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 10:50:32.107298 28373 catalog_manager.cc:5495] T 9d9f19b02ce64df5af2e9901b7418dce P afa84438b9224029840033fd30444c3f reported cstate change: term changed from 0 to 1, leader changed from <none> to afa84438b9224029840033fd30444c3f (127.20.80.193). New cstate: current_term: 1 leader_uuid: "afa84438b9224029840033fd30444c3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } health_report { overall_health: UNKNOWN } } }
I20260929 10:50:32.124717 28924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:32.121922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:34221 (local address 127.0.0.1:32954)
0929 10:50:32.122011 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:32.122025 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:32.122077 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:32.122241 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:32.122241 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:32.122243 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:50:32.122312 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:32.122316 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:32.122959 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:32.122961 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:32.123737 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:32.123739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:32.124119 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:32.124120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:32.124186 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:32.124209 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:50:32.124550 (+ 341us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:50:32.124561 (+ 11us) client_negotiation.cc:769] Sending connection context
0929 10:50:32.124591 (+ 30us) client_negotiation.cc:240] Negotiation successful
0929 10:50:32.124645 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260929 10:50:32.124734 28888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:32.121992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32954 (local address 127.20.80.193:34221)
0929 10:50:32.122044 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:32.122046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:32.122057 (+ 11us) server_negotiation.cc:407] Connection header received
0929 10:50:32.122151 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:32.122152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:32.122171 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:32.122235 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:50:32.122379 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:32.122874 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:32.123820 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:32.124033 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:32.124117 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:32.124322 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:50:32.124486 (+ 164us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:50:32.124565 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:32.124615 (+ 50us) server_negotiation.cc:299] Negotiation successful
0929 10:50:32.124634 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260929 10:50:32.125844 28674 raft_consensus.cc:1261] T 9d9f19b02ce64df5af2e9901b7418dce P db5e4a9496764d8d9a2f9bf9fa9df507 [term 1 FOLLOWER]: Refusing update from remote peer afa84438b9224029840033fd30444c3f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:32.125897 28815 raft_consensus.cc:1261] T 9d9f19b02ce64df5af2e9901b7418dce P 97c90aace77b4f56b9f8224b3688aba4 [term 1 FOLLOWER]: Refusing update from remote peer afa84438b9224029840033fd30444c3f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:32.126098 28931 consensus_queue.cc:1035] T 9d9f19b02ce64df5af2e9901b7418dce P afa84438b9224029840033fd30444c3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:50:32.126206 28894 consensus_queue.cc:1035] T 9d9f19b02ce64df5af2e9901b7418dce P afa84438b9224029840033fd30444c3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:50:32.126739 28898 tablet_replica.cc:439] T 9d9f19b02ce64df5af2e9901b7418dce P db5e4a9496764d8d9a2f9bf9fa9df507: TxnStatusTablet state changed. Reason: New leader afa84438b9224029840033fd30444c3f. Latest consensus state: current_term: 1 leader_uuid: "afa84438b9224029840033fd30444c3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } }
I20260929 10:50:32.126829 28898 tablet_replica.cc:442] T 9d9f19b02ce64df5af2e9901b7418dce P db5e4a9496764d8d9a2f9bf9fa9df507: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:50:32.127099 28900 tablet_replica.cc:439] T 9d9f19b02ce64df5af2e9901b7418dce P 97c90aace77b4f56b9f8224b3688aba4: TxnStatusTablet state changed. Reason: New leader afa84438b9224029840033fd30444c3f. Latest consensus state: current_term: 1 leader_uuid: "afa84438b9224029840033fd30444c3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } }
I20260929 10:50:32.127197 28900 tablet_replica.cc:442] T 9d9f19b02ce64df5af2e9901b7418dce P 97c90aace77b4f56b9f8224b3688aba4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:50:32.127789 28931 tablet_replica.cc:439] T 9d9f19b02ce64df5af2e9901b7418dce P afa84438b9224029840033fd30444c3f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "afa84438b9224029840033fd30444c3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } }
I20260929 10:50:32.127876 28931 tablet_replica.cc:442] T 9d9f19b02ce64df5af2e9901b7418dce P afa84438b9224029840033fd30444c3f: This TxnStatusTablet replica's current role is: LEADER
I20260929 10:50:32.127770 28894 tablet_replica.cc:439] T 9d9f19b02ce64df5af2e9901b7418dce P afa84438b9224029840033fd30444c3f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "afa84438b9224029840033fd30444c3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } }
I20260929 10:50:32.127945 28894 tablet_replica.cc:442] T 9d9f19b02ce64df5af2e9901b7418dce P afa84438b9224029840033fd30444c3f: This TxnStatusTablet replica's current role is: LEADER
I20260929 10:50:32.128119 28898 tablet_replica.cc:439] T 9d9f19b02ce64df5af2e9901b7418dce P db5e4a9496764d8d9a2f9bf9fa9df507: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "afa84438b9224029840033fd30444c3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } }
I20260929 10:50:32.128197 28898 tablet_replica.cc:442] T 9d9f19b02ce64df5af2e9901b7418dce P db5e4a9496764d8d9a2f9bf9fa9df507: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:50:32.128373 28900 tablet_replica.cc:439] T 9d9f19b02ce64df5af2e9901b7418dce P 97c90aace77b4f56b9f8224b3688aba4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "afa84438b9224029840033fd30444c3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } }
I20260929 10:50:32.128460 28900 tablet_replica.cc:442] T 9d9f19b02ce64df5af2e9901b7418dce P 97c90aace77b4f56b9f8224b3688aba4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:50:32.134364 28888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:32.131727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:58831 (local address 127.20.80.193:34221)
0929 10:50:32.131802 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:32.131804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:32.131846 (+ 42us) server_negotiation.cc:407] Connection header received
0929 10:50:32.131918 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:32.131919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:32.131929 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:32.131979 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:50:32.132069 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:32.132809 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:32.133626 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:32.133925 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:32.133995 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:32.134144 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:50:32.134177 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:50:32.134228 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:32.134276 (+ 48us) server_negotiation.cc:299] Negotiation successful
0929 10:50:32.134296 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 10:50:32.134382 28941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:32.131579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:34221 (local address 127.20.80.195:58831)
0929 10:50:32.131739 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:32.131766 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:32.131840 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:32.131987 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:32.131988 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:32.131989 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:50:32.132030 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:32.132031 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:32.132887 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:32.132888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:32.133543 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:32.133546 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:32.134004 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:32.134005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:32.134049 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:32.134063 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:50:32.134235 (+ 172us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:50:32.134237 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 10:50:32.134271 (+ 34us) client_negotiation.cc:240] Negotiation successful
0929 10:50:32.134307 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":90,"threads_started":1}
I20260929 10:50:32.136534 28674 raft_consensus.cc:1261] T fe19778390e84b8885646db70bd72991 P db5e4a9496764d8d9a2f9bf9fa9df507 [term 1 FOLLOWER]: Refusing update from remote peer 97c90aace77b4f56b9f8224b3688aba4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:32.136536 28533 raft_consensus.cc:1261] T fe19778390e84b8885646db70bd72991 P afa84438b9224029840033fd30444c3f [term 1 FOLLOWER]: Refusing update from remote peer 97c90aace77b4f56b9f8224b3688aba4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:32.136727 28900 consensus_queue.cc:1035] T fe19778390e84b8885646db70bd72991 P 97c90aace77b4f56b9f8224b3688aba4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:50:32.136837 28900 consensus_queue.cc:1035] T fe19778390e84b8885646db70bd72991 P 97c90aace77b4f56b9f8224b3688aba4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:50:32.145386 28889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:32.142841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:49061 (local address 127.20.80.195:41667)
0929 10:50:32.142878 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:32.142880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:32.142927 (+ 47us) server_negotiation.cc:407] Connection header received
0929 10:50:32.142998 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:32.142999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:32.143013 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:32.143061 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:50:32.143199 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:32.143675 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:32.144569 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:32.144843 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:32.144913 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:32.145117 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:50:32.145150 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:50:32.145221 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:32.145274 (+ 53us) server_negotiation.cc:299] Negotiation successful
0929 10:50:32.145301 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 10:50:32.145403 28946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:32.142704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:41667 (local address 127.20.80.193:49061)
0929 10:50:32.142869 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:32.142882 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:32.142942 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:32.143062 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:32.143062 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:32.143064 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:50:32.143130 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:32.143132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:32.143766 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:32.143767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:32.144460 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:32.144463 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:32.144930 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:32.144931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:32.145015 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:32.145031 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:50:32.145221 (+ 190us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:50:32.145224 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:50:32.145271 (+ 47us) client_negotiation.cc:240] Negotiation successful
0929 10:50:32.145314 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":79,"threads_started":1}
I20260929 10:50:32.155062 28375 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:50:32.155195 28375 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:50:32.156279 28375 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c04d4b30fd884a0f83bf748dc15c3a26: Sending DeleteTablet for 3 replicas of tablet fe19778390e84b8885646db70bd72991
I20260929 10:50:32.156529 28654 tablet_service.cc:1514] Processing DeleteTablet for tablet fe19778390e84b8885646db70bd72991 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:32 UTC) from {username='oryx', principal='oryx/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:48180
I20260929 10:50:32.156577 28795 tablet_service.cc:1514] Processing DeleteTablet for tablet fe19778390e84b8885646db70bd72991 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:32 UTC) from {username='oryx', principal='oryx/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:55526
I20260929 10:50:32.156638 28912 tablet_replica.cc:330] T fe19778390e84b8885646db70bd72991 P db5e4a9496764d8d9a2f9bf9fa9df507: stopping tablet replica
I20260929 10:50:32.156644 28513 tablet_service.cc:1514] Processing DeleteTablet for tablet fe19778390e84b8885646db70bd72991 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:32 UTC) from {username='oryx', principal='oryx/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:32950
I20260929 10:50:32.156669 28913 tablet_replica.cc:330] T fe19778390e84b8885646db70bd72991 P 97c90aace77b4f56b9f8224b3688aba4: stopping tablet replica
I20260929 10:50:32.156698 28912 raft_consensus.cc:2229] T fe19778390e84b8885646db70bd72991 P db5e4a9496764d8d9a2f9bf9fa9df507 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:50:32.156718 28913 raft_consensus.cc:2229] T fe19778390e84b8885646db70bd72991 P 97c90aace77b4f56b9f8224b3688aba4 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:50:32.156742 28912 raft_consensus.cc:2258] T fe19778390e84b8885646db70bd72991 P db5e4a9496764d8d9a2f9bf9fa9df507 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:32.156781 28913 raft_consensus.cc:2258] T fe19778390e84b8885646db70bd72991 P 97c90aace77b4f56b9f8224b3688aba4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:32.156809 28914 tablet_replica.cc:330] T fe19778390e84b8885646db70bd72991 P afa84438b9224029840033fd30444c3f: stopping tablet replica
I20260929 10:50:32.156854 28914 raft_consensus.cc:2229] T fe19778390e84b8885646db70bd72991 P afa84438b9224029840033fd30444c3f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:50:32.156890 28914 raft_consensus.cc:2258] T fe19778390e84b8885646db70bd72991 P afa84438b9224029840033fd30444c3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:32.157105 28913 ts_tablet_manager.cc:1905] T fe19778390e84b8885646db70bd72991 P 97c90aace77b4f56b9f8224b3688aba4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:32.157189 28914 ts_tablet_manager.cc:1905] T fe19778390e84b8885646db70bd72991 P afa84438b9224029840033fd30444c3f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:32.158845 28912 ts_tablet_manager.cc:1905] T fe19778390e84b8885646db70bd72991 P db5e4a9496764d8d9a2f9bf9fa9df507: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:32.159008 28913 ts_tablet_manager.cc:1918] T fe19778390e84b8885646db70bd72991 P 97c90aace77b4f56b9f8224b3688aba4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:50:32.159062 28913 log.cc:1196] T fe19778390e84b8885646db70bd72991 P 97c90aace77b4f56b9f8224b3688aba4: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-2/wal/wals/fe19778390e84b8885646db70bd72991
I20260929 10:50:32.159183 28914 ts_tablet_manager.cc:1918] T fe19778390e84b8885646db70bd72991 P afa84438b9224029840033fd30444c3f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:50:32.159240 28914 log.cc:1196] T fe19778390e84b8885646db70bd72991 P afa84438b9224029840033fd30444c3f: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-0/wal/wals/fe19778390e84b8885646db70bd72991
I20260929 10:50:32.159283 28913 ts_tablet_manager.cc:1939] T fe19778390e84b8885646db70bd72991 P 97c90aace77b4f56b9f8224b3688aba4: Deleting consensus metadata
I20260929 10:50:32.159498 28914 ts_tablet_manager.cc:1939] T fe19778390e84b8885646db70bd72991 P afa84438b9224029840033fd30444c3f: Deleting consensus metadata
I20260929 10:50:32.159914 28361 catalog_manager.cc:4841] TS afa84438b9224029840033fd30444c3f (127.20.80.193:34221): tablet fe19778390e84b8885646db70bd72991 (table test-table [id=fa2153933bc34b4d9a3fa98e9aac237a]) successfully deleted
I20260929 10:50:32.160266 28360 catalog_manager.cc:4841] TS 97c90aace77b4f56b9f8224b3688aba4 (127.20.80.195:41667): tablet fe19778390e84b8885646db70bd72991 (table test-table [id=fa2153933bc34b4d9a3fa98e9aac237a]) successfully deleted
I20260929 10:50:32.160615 28912 ts_tablet_manager.cc:1918] T fe19778390e84b8885646db70bd72991 P db5e4a9496764d8d9a2f9bf9fa9df507: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:50:32.160686 28912 log.cc:1196] T fe19778390e84b8885646db70bd72991 P db5e4a9496764d8d9a2f9bf9fa9df507: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-1/wal/wals/fe19778390e84b8885646db70bd72991
I20260929 10:50:32.160912 28912 ts_tablet_manager.cc:1939] T fe19778390e84b8885646db70bd72991 P db5e4a9496764d8d9a2f9bf9fa9df507: Deleting consensus metadata
I20260929 10:50:32.161230 28359 catalog_manager.cc:4841] TS db5e4a9496764d8d9a2f9bf9fa9df507 (127.20.80.194:33969): tablet fe19778390e84b8885646db70bd72991 (table test-table [id=fa2153933bc34b4d9a3fa98e9aac237a]) successfully deleted
Sep 29 10:50:32 dist-test-slave-vqch krb5kdc[28329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 10:50:32 dist-test-slave-vqch krb5kdc[28329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:32 dist-test-slave-vqch krb5kdc[28329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
W20260929 10:50:32.184935 28925 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:32.178879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57686 (local address 127.20.80.254:42259)
0929 10:50:32.178953 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:32.178955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:32.179103 (+ 148us) server_negotiation.cc:407] Connection header received
0929 10:50:32.179186 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:32.179187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:32.179197 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:32.179248 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:32.179953 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:32.180464 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:32.181607 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:32.181803 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:32.181859 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:32.184866 (+ 3007us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57686: BlockingRecv error: recv got EOF from 127.0.0.1:57686 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 29 10:50:32 dist-test-slave-vqch krb5kdc[28329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM
I20260929 10:50:32.193018 28925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:32.185848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57688 (local address 127.20.80.254:42259)
0929 10:50:32.185883 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:32.185885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:32.185991 (+ 106us) server_negotiation.cc:407] Connection header received
0929 10:50:32.186067 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:32.186068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:32.186078 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:32.186140 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:32.186610 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:32.187078 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:32.187781 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:32.187997 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:32.188071 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:32.190521 (+ 2450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:32.190523 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:32.190523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:32.190530 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:32.191863 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:32.192200 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:32.192201 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:32.192201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:32.192261 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:32.192492 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:32.192492 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:32.192493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:32.192607 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:32.192710 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:32.192812 (+ 102us) server_negotiation.cc:299] Negotiation successful
0929 10:50:32.192894 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260929 10:50:32.194563 28375 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57688:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:50:32.194738 28375 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:50:32.196048 28654 tablet_service.cc:1467] Processing CreateTablet for tablet 2d182a5f821f441abbe44b766aea8eef (DEFAULT_TABLE table=test-table [id=2acb4de75e114613b74087f4e861c211]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:32.196080 28513 tablet_service.cc:1467] Processing CreateTablet for tablet 2d182a5f821f441abbe44b766aea8eef (DEFAULT_TABLE table=test-table [id=2acb4de75e114613b74087f4e861c211]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:32.196080 28795 tablet_service.cc:1467] Processing CreateTablet for tablet 2d182a5f821f441abbe44b766aea8eef (DEFAULT_TABLE table=test-table [id=2acb4de75e114613b74087f4e861c211]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:32.196187 28654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d182a5f821f441abbe44b766aea8eef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:32.196204 28513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d182a5f821f441abbe44b766aea8eef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:32.196204 28795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d182a5f821f441abbe44b766aea8eef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:32.197558 28891 tablet_bootstrap.cc:492] T 2d182a5f821f441abbe44b766aea8eef P afa84438b9224029840033fd30444c3f: Bootstrap starting.
I20260929 10:50:32.197589 28896 tablet_bootstrap.cc:492] T 2d182a5f821f441abbe44b766aea8eef P db5e4a9496764d8d9a2f9bf9fa9df507: Bootstrap starting.
I20260929 10:50:32.197924 28891 tablet_bootstrap.cc:654] T 2d182a5f821f441abbe44b766aea8eef P afa84438b9224029840033fd30444c3f: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:32.197969 28896 tablet_bootstrap.cc:654] T 2d182a5f821f441abbe44b766aea8eef P db5e4a9496764d8d9a2f9bf9fa9df507: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:32.198208 28890 tablet_bootstrap.cc:492] T 2d182a5f821f441abbe44b766aea8eef P 97c90aace77b4f56b9f8224b3688aba4: Bootstrap starting.
I20260929 10:50:32.198588 28890 tablet_bootstrap.cc:654] T 2d182a5f821f441abbe44b766aea8eef P 97c90aace77b4f56b9f8224b3688aba4: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:32.198616 28896 tablet_bootstrap.cc:492] T 2d182a5f821f441abbe44b766aea8eef P db5e4a9496764d8d9a2f9bf9fa9df507: No bootstrap required, opened a new log
I20260929 10:50:32.198679 28896 ts_tablet_manager.cc:1397] T 2d182a5f821f441abbe44b766aea8eef P db5e4a9496764d8d9a2f9bf9fa9df507: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:32.198837 28896 raft_consensus.cc:348] T 2d182a5f821f441abbe44b766aea8eef P db5e4a9496764d8d9a2f9bf9fa9df507 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } }
I20260929 10:50:32.198918 28896 raft_consensus.cc:374] T 2d182a5f821f441abbe44b766aea8eef P db5e4a9496764d8d9a2f9bf9fa9df507 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:32.198948 28896 raft_consensus.cc:729] T 2d182a5f821f441abbe44b766aea8eef P db5e4a9496764d8d9a2f9bf9fa9df507 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db5e4a9496764d8d9a2f9bf9fa9df507, State: Initialized, Role: FOLLOWER
I20260929 10:50:32.198999 28891 tablet_bootstrap.cc:492] T 2d182a5f821f441abbe44b766aea8eef P afa84438b9224029840033fd30444c3f: No bootstrap required, opened a new log
I20260929 10:50:32.198985 28896 consensus_queue.cc:260] T 2d182a5f821f441abbe44b766aea8eef P db5e4a9496764d8d9a2f9bf9fa9df507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } }
I20260929 10:50:32.199043 28891 ts_tablet_manager.cc:1397] T 2d182a5f821f441abbe44b766aea8eef P afa84438b9224029840033fd30444c3f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:50:32.199077 28896 ts_tablet_manager.cc:1428] T 2d182a5f821f441abbe44b766aea8eef P db5e4a9496764d8d9a2f9bf9fa9df507: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:32.199178 28891 raft_consensus.cc:348] T 2d182a5f821f441abbe44b766aea8eef P afa84438b9224029840033fd30444c3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } }
I20260929 10:50:32.199251 28891 raft_consensus.cc:374] T 2d182a5f821f441abbe44b766aea8eef P afa84438b9224029840033fd30444c3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:32.199301 28891 raft_consensus.cc:729] T 2d182a5f821f441abbe44b766aea8eef P afa84438b9224029840033fd30444c3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afa84438b9224029840033fd30444c3f, State: Initialized, Role: FOLLOWER
I20260929 10:50:32.199342 28891 consensus_queue.cc:260] T 2d182a5f821f441abbe44b766aea8eef P afa84438b9224029840033fd30444c3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } }
I20260929 10:50:32.199443 28891 ts_tablet_manager.cc:1428] T 2d182a5f821f441abbe44b766aea8eef P afa84438b9224029840033fd30444c3f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:32.199666 28890 tablet_bootstrap.cc:492] T 2d182a5f821f441abbe44b766aea8eef P 97c90aace77b4f56b9f8224b3688aba4: No bootstrap required, opened a new log
I20260929 10:50:32.199721 28890 ts_tablet_manager.cc:1397] T 2d182a5f821f441abbe44b766aea8eef P 97c90aace77b4f56b9f8224b3688aba4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:50:32.199841 28890 raft_consensus.cc:348] T 2d182a5f821f441abbe44b766aea8eef P 97c90aace77b4f56b9f8224b3688aba4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } }
I20260929 10:50:32.199911 28890 raft_consensus.cc:374] T 2d182a5f821f441abbe44b766aea8eef P 97c90aace77b4f56b9f8224b3688aba4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:32.199942 28890 raft_consensus.cc:729] T 2d182a5f821f441abbe44b766aea8eef P 97c90aace77b4f56b9f8224b3688aba4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97c90aace77b4f56b9f8224b3688aba4, State: Initialized, Role: FOLLOWER
I20260929 10:50:32.199994 28890 consensus_queue.cc:260] T 2d182a5f821f441abbe44b766aea8eef P 97c90aace77b4f56b9f8224b3688aba4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } }
I20260929 10:50:32.200088 28890 ts_tablet_manager.cc:1428] T 2d182a5f821f441abbe44b766aea8eef P 97c90aace77b4f56b9f8224b3688aba4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:32.206802 28894 raft_consensus.cc:482] T 2d182a5f821f441abbe44b766aea8eef P afa84438b9224029840033fd30444c3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:50:32.206885 28894 raft_consensus.cc:504] T 2d182a5f821f441abbe44b766aea8eef P afa84438b9224029840033fd30444c3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } }
I20260929 10:50:32.207029 28894 leader_election.cc:288] T 2d182a5f821f441abbe44b766aea8eef P afa84438b9224029840033fd30444c3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db5e4a9496764d8d9a2f9bf9fa9df507 (127.20.80.194:33969), 97c90aace77b4f56b9f8224b3688aba4 (127.20.80.195:41667)
I20260929 10:50:32.207243 28815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d182a5f821f441abbe44b766aea8eef" candidate_uuid: "afa84438b9224029840033fd30444c3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97c90aace77b4f56b9f8224b3688aba4" is_pre_election: true
I20260929 10:50:32.207337 28815 raft_consensus.cc:2454] T 2d182a5f821f441abbe44b766aea8eef P 97c90aace77b4f56b9f8224b3688aba4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afa84438b9224029840033fd30444c3f in term 0.
I20260929 10:50:32.207305 28674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d182a5f821f441abbe44b766aea8eef" candidate_uuid: "afa84438b9224029840033fd30444c3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" is_pre_election: true
I20260929 10:50:32.207414 28674 raft_consensus.cc:2454] T 2d182a5f821f441abbe44b766aea8eef P db5e4a9496764d8d9a2f9bf9fa9df507 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afa84438b9224029840033fd30444c3f in term 0.
I20260929 10:50:32.207508 28466 leader_election.cc:302] T 2d182a5f821f441abbe44b766aea8eef P afa84438b9224029840033fd30444c3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97c90aace77b4f56b9f8224b3688aba4, afa84438b9224029840033fd30444c3f; no voters:
I20260929 10:50:32.207600 28894 raft_consensus.cc:2781] T 2d182a5f821f441abbe44b766aea8eef P afa84438b9224029840033fd30444c3f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:50:32.207646 28894 raft_consensus.cc:482] T 2d182a5f821f441abbe44b766aea8eef P afa84438b9224029840033fd30444c3f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:50:32.207677 28894 raft_consensus.cc:3037] T 2d182a5f821f441abbe44b766aea8eef P afa84438b9224029840033fd30444c3f [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:32.208633 28894 raft_consensus.cc:504] T 2d182a5f821f441abbe44b766aea8eef P afa84438b9224029840033fd30444c3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } }
I20260929 10:50:32.208751 28894 leader_election.cc:288] T 2d182a5f821f441abbe44b766aea8eef P afa84438b9224029840033fd30444c3f [CANDIDATE]: Term 1 election: Requested vote from peers db5e4a9496764d8d9a2f9bf9fa9df507 (127.20.80.194:33969), 97c90aace77b4f56b9f8224b3688aba4 (127.20.80.195:41667)
I20260929 10:50:32.213407 28674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d182a5f821f441abbe44b766aea8eef" candidate_uuid: "afa84438b9224029840033fd30444c3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507"
I20260929 10:50:32.213513 28674 raft_consensus.cc:3037] T 2d182a5f821f441abbe44b766aea8eef P db5e4a9496764d8d9a2f9bf9fa9df507 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:32.214581 28674 raft_consensus.cc:2454] T 2d182a5f821f441abbe44b766aea8eef P db5e4a9496764d8d9a2f9bf9fa9df507 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afa84438b9224029840033fd30444c3f in term 1.
I20260929 10:50:32.214772 28465 leader_election.cc:302] T 2d182a5f821f441abbe44b766aea8eef P afa84438b9224029840033fd30444c3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: afa84438b9224029840033fd30444c3f, db5e4a9496764d8d9a2f9bf9fa9df507; no voters:
I20260929 10:50:32.214970 28894 raft_consensus.cc:2781] T 2d182a5f821f441abbe44b766aea8eef P afa84438b9224029840033fd30444c3f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:32.215030 28894 raft_consensus.cc:686] T 2d182a5f821f441abbe44b766aea8eef P afa84438b9224029840033fd30444c3f [term 1 LEADER]: Becoming Leader. State: Replica: afa84438b9224029840033fd30444c3f, State: Running, Role: LEADER
I20260929 10:50:32.215131 28894 consensus_queue.cc:237] T 2d182a5f821f441abbe44b766aea8eef P afa84438b9224029840033fd30444c3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } }
I20260929 10:50:32.217329 28815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d182a5f821f441abbe44b766aea8eef" candidate_uuid: "afa84438b9224029840033fd30444c3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97c90aace77b4f56b9f8224b3688aba4"
I20260929 10:50:32.217437 28815 raft_consensus.cc:3037] T 2d182a5f821f441abbe44b766aea8eef P 97c90aace77b4f56b9f8224b3688aba4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:32.217551 28373 catalog_manager.cc:5495] T 2d182a5f821f441abbe44b766aea8eef P afa84438b9224029840033fd30444c3f reported cstate change: term changed from 0 to 1, leader changed from <none> to afa84438b9224029840033fd30444c3f (127.20.80.193). New cstate: current_term: 1 leader_uuid: "afa84438b9224029840033fd30444c3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } health_report { overall_health: UNKNOWN } } }
I20260929 10:50:32.218282 28815 raft_consensus.cc:2454] T 2d182a5f821f441abbe44b766aea8eef P 97c90aace77b4f56b9f8224b3688aba4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afa84438b9224029840033fd30444c3f in term 1.
I20260929 10:50:32.225127 28888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:32.222500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32958 (local address 127.20.80.193:34221)
0929 10:50:32.222547 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:32.222549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:32.222561 (+ 12us) server_negotiation.cc:407] Connection header received
0929 10:50:32.222586 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:32.222587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:32.222600 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:32.222669 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:50:32.222756 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:32.223513 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:32.224300 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:32.224594 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:32.224671 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:32.224793 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:50:32.224830 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:50:32.224899 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:32.225005 (+ 106us) server_negotiation.cc:299] Negotiation successful
0929 10:50:32.225027 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 10:50:32.225987 28674 raft_consensus.cc:1261] T 2d182a5f821f441abbe44b766aea8eef P db5e4a9496764d8d9a2f9bf9fa9df507 [term 1 FOLLOWER]: Refusing update from remote peer afa84438b9224029840033fd30444c3f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:32.226039 28815 raft_consensus.cc:1261] T 2d182a5f821f441abbe44b766aea8eef P 97c90aace77b4f56b9f8224b3688aba4 [term 1 FOLLOWER]: Refusing update from remote peer afa84438b9224029840033fd30444c3f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:32.226198 28894 consensus_queue.cc:1035] T 2d182a5f821f441abbe44b766aea8eef P afa84438b9224029840033fd30444c3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:50:32.226321 28894 consensus_queue.cc:1035] T 2d182a5f821f441abbe44b766aea8eef P afa84438b9224029840033fd30444c3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:50:32.229476 28373 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:50:32.229573 28373 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:50:32.230206 28373 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c04d4b30fd884a0f83bf748dc15c3a26: Sending DeleteTablet for 3 replicas of tablet 2d182a5f821f441abbe44b766aea8eef
I20260929 10:50:32.230535 28654 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d182a5f821f441abbe44b766aea8eef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:32 UTC) from {username='oryx', principal='oryx/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:48180
I20260929 10:50:32.230638 28912 tablet_replica.cc:330] T 2d182a5f821f441abbe44b766aea8eef P db5e4a9496764d8d9a2f9bf9fa9df507: stopping tablet replica
I20260929 10:50:32.230700 28912 raft_consensus.cc:2229] T 2d182a5f821f441abbe44b766aea8eef P db5e4a9496764d8d9a2f9bf9fa9df507 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:50:32.230695 28513 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d182a5f821f441abbe44b766aea8eef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:32 UTC) from {username='oryx', principal='oryx/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:32950
I20260929 10:50:32.230751 28912 raft_consensus.cc:2258] T 2d182a5f821f441abbe44b766aea8eef P db5e4a9496764d8d9a2f9bf9fa9df507 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:32.230779 28914 tablet_replica.cc:330] T 2d182a5f821f441abbe44b766aea8eef P afa84438b9224029840033fd30444c3f: stopping tablet replica
I20260929 10:50:32.230729 28373 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57688:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 10:50:32.230832 28914 raft_consensus.cc:2229] T 2d182a5f821f441abbe44b766aea8eef P afa84438b9224029840033fd30444c3f [term 1 LEADER]: Raft consensus shutting down.
W20260929 10:50:32.230860 28373 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:50:32.230914 28914 raft_consensus.cc:2258] T 2d182a5f821f441abbe44b766aea8eef P afa84438b9224029840033fd30444c3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:32.231065 28795 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d182a5f821f441abbe44b766aea8eef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:32 UTC) from {username='oryx', principal='oryx/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:55526
I20260929 10:50:32.231084 28912 ts_tablet_manager.cc:1905] T 2d182a5f821f441abbe44b766aea8eef P db5e4a9496764d8d9a2f9bf9fa9df507: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:32.231146 28913 tablet_replica.cc:330] T 2d182a5f821f441abbe44b766aea8eef P 97c90aace77b4f56b9f8224b3688aba4: stopping tablet replica
I20260929 10:50:32.231199 28913 raft_consensus.cc:2229] T 2d182a5f821f441abbe44b766aea8eef P 97c90aace77b4f56b9f8224b3688aba4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:50:32.231235 28913 raft_consensus.cc:2258] T 2d182a5f821f441abbe44b766aea8eef P 97c90aace77b4f56b9f8224b3688aba4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:32.232053 28795 tablet_service.cc:1467] Processing CreateTablet for tablet 4d151670899841f89b99768f3e6a042c (DEFAULT_TABLE table=test-table [id=d781f2ce77234d86a7596ee564f41f89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:32.232048 28654 tablet_service.cc:1467] Processing CreateTablet for tablet 4d151670899841f89b99768f3e6a042c (DEFAULT_TABLE table=test-table [id=d781f2ce77234d86a7596ee564f41f89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:32.232161 28654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d151670899841f89b99768f3e6a042c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:32.232162 28795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d151670899841f89b99768f3e6a042c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:32.232671 28912 ts_tablet_manager.cc:1918] T 2d182a5f821f441abbe44b766aea8eef P db5e4a9496764d8d9a2f9bf9fa9df507: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:50:32.232722 28912 log.cc:1196] T 2d182a5f821f441abbe44b766aea8eef P db5e4a9496764d8d9a2f9bf9fa9df507: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-1/wal/wals/2d182a5f821f441abbe44b766aea8eef
I20260929 10:50:32.232954 28912 ts_tablet_manager.cc:1939] T 2d182a5f821f441abbe44b766aea8eef P db5e4a9496764d8d9a2f9bf9fa9df507: Deleting consensus metadata
I20260929 10:50:32.233378 28359 catalog_manager.cc:4841] TS db5e4a9496764d8d9a2f9bf9fa9df507 (127.20.80.194:33969): tablet 2d182a5f821f441abbe44b766aea8eef (table test-table [id=2acb4de75e114613b74087f4e861c211]) successfully deleted
I20260929 10:50:32.233448 28890 tablet_bootstrap.cc:492] T 4d151670899841f89b99768f3e6a042c P 97c90aace77b4f56b9f8224b3688aba4: Bootstrap starting.
I20260929 10:50:32.233731 28890 tablet_bootstrap.cc:654] T 4d151670899841f89b99768f3e6a042c P 97c90aace77b4f56b9f8224b3688aba4: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:32.234164 28513 tablet_service.cc:1467] Processing CreateTablet for tablet 4d151670899841f89b99768f3e6a042c (DEFAULT_TABLE table=test-table [id=d781f2ce77234d86a7596ee564f41f89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:32.234269 28513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d151670899841f89b99768f3e6a042c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:32.234283 28890 tablet_bootstrap.cc:492] T 4d151670899841f89b99768f3e6a042c P 97c90aace77b4f56b9f8224b3688aba4: No bootstrap required, opened a new log
I20260929 10:50:32.234321 28890 ts_tablet_manager.cc:1397] T 4d151670899841f89b99768f3e6a042c P 97c90aace77b4f56b9f8224b3688aba4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260929 10:50:32.234400 28890 raft_consensus.cc:348] T 4d151670899841f89b99768f3e6a042c P 97c90aace77b4f56b9f8224b3688aba4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } }
I20260929 10:50:32.234442 28890 raft_consensus.cc:374] T 4d151670899841f89b99768f3e6a042c P 97c90aace77b4f56b9f8224b3688aba4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:32.234454 28890 raft_consensus.cc:729] T 4d151670899841f89b99768f3e6a042c P 97c90aace77b4f56b9f8224b3688aba4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97c90aace77b4f56b9f8224b3688aba4, State: Initialized, Role: FOLLOWER
I20260929 10:50:32.234513 28890 consensus_queue.cc:260] T 4d151670899841f89b99768f3e6a042c P 97c90aace77b4f56b9f8224b3688aba4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } }
I20260929 10:50:32.234603 28890 ts_tablet_manager.cc:1428] T 4d151670899841f89b99768f3e6a042c P 97c90aace77b4f56b9f8224b3688aba4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:32.234897 28896 tablet_bootstrap.cc:492] T 4d151670899841f89b99768f3e6a042c P db5e4a9496764d8d9a2f9bf9fa9df507: Bootstrap starting.
I20260929 10:50:32.235253 28896 tablet_bootstrap.cc:654] T 4d151670899841f89b99768f3e6a042c P db5e4a9496764d8d9a2f9bf9fa9df507: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:32.235532 28891 tablet_bootstrap.cc:492] T 4d151670899841f89b99768f3e6a042c P afa84438b9224029840033fd30444c3f: Bootstrap starting.
I20260929 10:50:32.235962 28891 tablet_bootstrap.cc:654] T 4d151670899841f89b99768f3e6a042c P afa84438b9224029840033fd30444c3f: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:32.236178 28896 tablet_bootstrap.cc:492] T 4d151670899841f89b99768f3e6a042c P db5e4a9496764d8d9a2f9bf9fa9df507: No bootstrap required, opened a new log
I20260929 10:50:32.236243 28896 ts_tablet_manager.cc:1397] T 4d151670899841f89b99768f3e6a042c P db5e4a9496764d8d9a2f9bf9fa9df507: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:32.236431 28896 raft_consensus.cc:348] T 4d151670899841f89b99768f3e6a042c P db5e4a9496764d8d9a2f9bf9fa9df507 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } }
I20260929 10:50:32.236505 28896 raft_consensus.cc:374] T 4d151670899841f89b99768f3e6a042c P db5e4a9496764d8d9a2f9bf9fa9df507 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:32.236532 28896 raft_consensus.cc:729] T 4d151670899841f89b99768f3e6a042c P db5e4a9496764d8d9a2f9bf9fa9df507 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db5e4a9496764d8d9a2f9bf9fa9df507, State: Initialized, Role: FOLLOWER
I20260929 10:50:32.236573 28896 consensus_queue.cc:260] T 4d151670899841f89b99768f3e6a042c P db5e4a9496764d8d9a2f9bf9fa9df507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } }
I20260929 10:50:32.236668 28896 ts_tablet_manager.cc:1428] T 4d151670899841f89b99768f3e6a042c P db5e4a9496764d8d9a2f9bf9fa9df507: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:32.236749 28891 tablet_bootstrap.cc:492] T 4d151670899841f89b99768f3e6a042c P afa84438b9224029840033fd30444c3f: No bootstrap required, opened a new log
I20260929 10:50:32.236785 28891 ts_tablet_manager.cc:1397] T 4d151670899841f89b99768f3e6a042c P afa84438b9224029840033fd30444c3f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:32.236941 28891 raft_consensus.cc:348] T 4d151670899841f89b99768f3e6a042c P afa84438b9224029840033fd30444c3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } }
I20260929 10:50:32.237053 28891 raft_consensus.cc:374] T 4d151670899841f89b99768f3e6a042c P afa84438b9224029840033fd30444c3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:32.237082 28891 raft_consensus.cc:729] T 4d151670899841f89b99768f3e6a042c P afa84438b9224029840033fd30444c3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afa84438b9224029840033fd30444c3f, State: Initialized, Role: FOLLOWER
I20260929 10:50:32.237120 28891 consensus_queue.cc:260] T 4d151670899841f89b99768f3e6a042c P afa84438b9224029840033fd30444c3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } }
I20260929 10:50:32.237231 28891 ts_tablet_manager.cc:1428] T 4d151670899841f89b99768f3e6a042c P afa84438b9224029840033fd30444c3f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:32.237340 28914 ts_tablet_manager.cc:1905] T 2d182a5f821f441abbe44b766aea8eef P afa84438b9224029840033fd30444c3f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:32.238693 28913 ts_tablet_manager.cc:1905] T 2d182a5f821f441abbe44b766aea8eef P 97c90aace77b4f56b9f8224b3688aba4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:32.238875 28914 ts_tablet_manager.cc:1918] T 2d182a5f821f441abbe44b766aea8eef P afa84438b9224029840033fd30444c3f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:50:32.238956 28914 log.cc:1196] T 2d182a5f821f441abbe44b766aea8eef P afa84438b9224029840033fd30444c3f: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-0/wal/wals/2d182a5f821f441abbe44b766aea8eef
I20260929 10:50:32.239154 28914 ts_tablet_manager.cc:1939] T 2d182a5f821f441abbe44b766aea8eef P afa84438b9224029840033fd30444c3f: Deleting consensus metadata
I20260929 10:50:32.239465 28361 catalog_manager.cc:4841] TS afa84438b9224029840033fd30444c3f (127.20.80.193:34221): tablet 2d182a5f821f441abbe44b766aea8eef (table test-table [id=2acb4de75e114613b74087f4e861c211]) successfully deleted
I20260929 10:50:32.240136 28913 ts_tablet_manager.cc:1918] T 2d182a5f821f441abbe44b766aea8eef P 97c90aace77b4f56b9f8224b3688aba4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:50:32.240206 28913 log.cc:1196] T 2d182a5f821f441abbe44b766aea8eef P 97c90aace77b4f56b9f8224b3688aba4: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-2/wal/wals/2d182a5f821f441abbe44b766aea8eef
I20260929 10:50:32.240417 28913 ts_tablet_manager.cc:1939] T 2d182a5f821f441abbe44b766aea8eef P 97c90aace77b4f56b9f8224b3688aba4: Deleting consensus metadata
I20260929 10:50:32.240734 28360 catalog_manager.cc:4841] TS 97c90aace77b4f56b9f8224b3688aba4 (127.20.80.195:41667): tablet 2d182a5f821f441abbe44b766aea8eef (table test-table [id=2acb4de75e114613b74087f4e861c211]) successfully deleted
I20260929 10:50:32.261575 28931 raft_consensus.cc:482] T 4d151670899841f89b99768f3e6a042c P afa84438b9224029840033fd30444c3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:50:32.261664 28931 raft_consensus.cc:504] T 4d151670899841f89b99768f3e6a042c P afa84438b9224029840033fd30444c3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } }
I20260929 10:50:32.261857 28931 leader_election.cc:288] T 4d151670899841f89b99768f3e6a042c P afa84438b9224029840033fd30444c3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db5e4a9496764d8d9a2f9bf9fa9df507 (127.20.80.194:33969), 97c90aace77b4f56b9f8224b3688aba4 (127.20.80.195:41667)
I20260929 10:50:32.262065 28674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d151670899841f89b99768f3e6a042c" candidate_uuid: "afa84438b9224029840033fd30444c3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" is_pre_election: true
I20260929 10:50:32.262102 28815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d151670899841f89b99768f3e6a042c" candidate_uuid: "afa84438b9224029840033fd30444c3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97c90aace77b4f56b9f8224b3688aba4" is_pre_election: true
I20260929 10:50:32.262179 28674 raft_consensus.cc:2454] T 4d151670899841f89b99768f3e6a042c P db5e4a9496764d8d9a2f9bf9fa9df507 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afa84438b9224029840033fd30444c3f in term 0.
I20260929 10:50:32.262199 28815 raft_consensus.cc:2454] T 4d151670899841f89b99768f3e6a042c P 97c90aace77b4f56b9f8224b3688aba4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afa84438b9224029840033fd30444c3f in term 0.
I20260929 10:50:32.262387 28466 leader_election.cc:302] T 4d151670899841f89b99768f3e6a042c P afa84438b9224029840033fd30444c3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97c90aace77b4f56b9f8224b3688aba4, afa84438b9224029840033fd30444c3f; no voters:
I20260929 10:50:32.262471 28931 raft_consensus.cc:2781] T 4d151670899841f89b99768f3e6a042c P afa84438b9224029840033fd30444c3f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:50:32.262516 28931 raft_consensus.cc:482] T 4d151670899841f89b99768f3e6a042c P afa84438b9224029840033fd30444c3f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:50:32.262529 28931 raft_consensus.cc:3037] T 4d151670899841f89b99768f3e6a042c P afa84438b9224029840033fd30444c3f [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:32.263170 28931 raft_consensus.cc:504] T 4d151670899841f89b99768f3e6a042c P afa84438b9224029840033fd30444c3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } }
I20260929 10:50:32.263300 28931 leader_election.cc:288] T 4d151670899841f89b99768f3e6a042c P afa84438b9224029840033fd30444c3f [CANDIDATE]: Term 1 election: Requested vote from peers db5e4a9496764d8d9a2f9bf9fa9df507 (127.20.80.194:33969), 97c90aace77b4f56b9f8224b3688aba4 (127.20.80.195:41667)
I20260929 10:50:32.263423 28674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d151670899841f89b99768f3e6a042c" candidate_uuid: "afa84438b9224029840033fd30444c3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507"
I20260929 10:50:32.263499 28674 raft_consensus.cc:3037] T 4d151670899841f89b99768f3e6a042c P db5e4a9496764d8d9a2f9bf9fa9df507 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:32.263471 28815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d151670899841f89b99768f3e6a042c" candidate_uuid: "afa84438b9224029840033fd30444c3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97c90aace77b4f56b9f8224b3688aba4"
I20260929 10:50:32.263542 28815 raft_consensus.cc:3037] T 4d151670899841f89b99768f3e6a042c P 97c90aace77b4f56b9f8224b3688aba4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:32.264074 28674 raft_consensus.cc:2454] T 4d151670899841f89b99768f3e6a042c P db5e4a9496764d8d9a2f9bf9fa9df507 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afa84438b9224029840033fd30444c3f in term 1.
I20260929 10:50:32.264248 28465 leader_election.cc:302] T 4d151670899841f89b99768f3e6a042c P afa84438b9224029840033fd30444c3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: afa84438b9224029840033fd30444c3f, db5e4a9496764d8d9a2f9bf9fa9df507; no voters:
I20260929 10:50:32.264350 28931 raft_consensus.cc:2781] T 4d151670899841f89b99768f3e6a042c P afa84438b9224029840033fd30444c3f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:32.264353 28815 raft_consensus.cc:2454] T 4d151670899841f89b99768f3e6a042c P 97c90aace77b4f56b9f8224b3688aba4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afa84438b9224029840033fd30444c3f in term 1.
I20260929 10:50:32.264397 28931 raft_consensus.cc:686] T 4d151670899841f89b99768f3e6a042c P afa84438b9224029840033fd30444c3f [term 1 LEADER]: Becoming Leader. State: Replica: afa84438b9224029840033fd30444c3f, State: Running, Role: LEADER
I20260929 10:50:32.264451 28931 consensus_queue.cc:237] T 4d151670899841f89b99768f3e6a042c P afa84438b9224029840033fd30444c3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } }
I20260929 10:50:32.265204 28373 catalog_manager.cc:5495] T 4d151670899841f89b99768f3e6a042c P afa84438b9224029840033fd30444c3f reported cstate change: term changed from 0 to 1, leader changed from <none> to afa84438b9224029840033fd30444c3f (127.20.80.193). New cstate: current_term: 1 leader_uuid: "afa84438b9224029840033fd30444c3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa84438b9224029840033fd30444c3f" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 } health_report { overall_health: UNKNOWN } } }
I20260929 10:50:32.278371 28888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:32.276157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:40723 (local address 127.20.80.193:34221)
0929 10:50:32.276202 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:32.276208 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:32.276217 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:50:32.276273 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:32.276274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:32.276284 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:32.276336 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:50:32.276467 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:32.276934 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:32.277665 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:32.277862 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:32.277953 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:32.278091 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:50:32.278124 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:50:32.278207 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:32.278264 (+ 57us) server_negotiation.cc:299] Negotiation successful
0929 10:50:32.278285 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":29}
I20260929 10:50:32.278371 28946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:32.276106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:34221 (local address 127.20.80.193:40723)
0929 10:50:32.276163 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:32.276174 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:32.276220 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:32.276339 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:32.276340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:32.276340 (+ 0us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:50:32.276404 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:32.276405 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:32.277035 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:32.277037 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:32.277579 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:32.277581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:32.277952 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:32.277953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:32.278001 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:32.278007 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:50:32.278205 (+ 198us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:50:32.278207 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 10:50:32.278260 (+ 53us) client_negotiation.cc:240] Negotiation successful
0929 10:50:32.278305 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":6}
I20260929 10:50:32.280356 28815 raft_consensus.cc:1261] T 4d151670899841f89b99768f3e6a042c P 97c90aace77b4f56b9f8224b3688aba4 [term 1 FOLLOWER]: Refusing update from remote peer afa84438b9224029840033fd30444c3f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:32.280400 28674 raft_consensus.cc:1261] T 4d151670899841f89b99768f3e6a042c P db5e4a9496764d8d9a2f9bf9fa9df507 [term 1 FOLLOWER]: Refusing update from remote peer afa84438b9224029840033fd30444c3f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:32.280587 28931 consensus_queue.cc:1035] T 4d151670899841f89b99768f3e6a042c P afa84438b9224029840033fd30444c3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db5e4a9496764d8d9a2f9bf9fa9df507" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:50:32.280705 28938 consensus_queue.cc:1035] T 4d151670899841f89b99768f3e6a042c P afa84438b9224029840033fd30444c3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "97c90aace77b4f56b9f8224b3688aba4" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:50:32.295866 28373 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:50:32.295979 28373 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:50:32.296626 28373 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c04d4b30fd884a0f83bf748dc15c3a26: Sending DeleteTablet for 3 replicas of tablet 4d151670899841f89b99768f3e6a042c
I20260929 10:50:32.296911 28654 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d151670899841f89b99768f3e6a042c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:32 UTC) from {username='oryx', principal='oryx/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:48180
I20260929 10:50:32.297027 28912 tablet_replica.cc:330] T 4d151670899841f89b99768f3e6a042c P db5e4a9496764d8d9a2f9bf9fa9df507: stopping tablet replica
I20260929 10:50:32.297082 28912 raft_consensus.cc:2229] T 4d151670899841f89b99768f3e6a042c P db5e4a9496764d8d9a2f9bf9fa9df507 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:50:32.297122 28912 raft_consensus.cc:2258] T 4d151670899841f89b99768f3e6a042c P db5e4a9496764d8d9a2f9bf9fa9df507 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:32.296968 28795 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d151670899841f89b99768f3e6a042c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:32 UTC) from {username='oryx', principal='oryx/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:55526
I20260929 10:50:32.297257 28913 tablet_replica.cc:330] T 4d151670899841f89b99768f3e6a042c P 97c90aace77b4f56b9f8224b3688aba4: stopping tablet replica
I20260929 10:50:32.297276 28513 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d151670899841f89b99768f3e6a042c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:32 UTC) from {username='oryx', principal='oryx/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:32950
I20260929 10:50:32.297305 28913 raft_consensus.cc:2229] T 4d151670899841f89b99768f3e6a042c P 97c90aace77b4f56b9f8224b3688aba4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:50:32.297339 28913 raft_consensus.cc:2258] T 4d151670899841f89b99768f3e6a042c P 97c90aace77b4f56b9f8224b3688aba4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:32.297538 28912 ts_tablet_manager.cc:1905] T 4d151670899841f89b99768f3e6a042c P db5e4a9496764d8d9a2f9bf9fa9df507: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:32.297533 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 28447
I20260929 10:50:32.298971 28912 ts_tablet_manager.cc:1918] T 4d151670899841f89b99768f3e6a042c P db5e4a9496764d8d9a2f9bf9fa9df507: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:50:32.299027 28912 log.cc:1196] T 4d151670899841f89b99768f3e6a042c P db5e4a9496764d8d9a2f9bf9fa9df507: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-1/wal/wals/4d151670899841f89b99768f3e6a042c
I20260929 10:50:32.299232 28912 ts_tablet_manager.cc:1939] T 4d151670899841f89b99768f3e6a042c P db5e4a9496764d8d9a2f9bf9fa9df507: Deleting consensus metadata
I20260929 10:50:32.299284 28913 ts_tablet_manager.cc:1905] T 4d151670899841f89b99768f3e6a042c P 97c90aace77b4f56b9f8224b3688aba4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:32.299618 28359 catalog_manager.cc:4841] TS db5e4a9496764d8d9a2f9bf9fa9df507 (127.20.80.194:33969): tablet 4d151670899841f89b99768f3e6a042c (table test-table [id=d781f2ce77234d86a7596ee564f41f89]) successfully deleted
I20260929 10:50:32.300702 28913 ts_tablet_manager.cc:1918] T 4d151670899841f89b99768f3e6a042c P 97c90aace77b4f56b9f8224b3688aba4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:50:32.300783 28913 log.cc:1196] T 4d151670899841f89b99768f3e6a042c P 97c90aace77b4f56b9f8224b3688aba4: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790678996126930-20803-0/minicluster-data/ts-2/wal/wals/4d151670899841f89b99768f3e6a042c
I20260929 10:50:32.301061 28913 ts_tablet_manager.cc:1939] T 4d151670899841f89b99768f3e6a042c P 97c90aace77b4f56b9f8224b3688aba4: Deleting consensus metadata
I20260929 10:50:32.301399 28360 catalog_manager.cc:4841] TS 97c90aace77b4f56b9f8224b3688aba4 (127.20.80.195:41667): tablet 4d151670899841f89b99768f3e6a042c (table test-table [id=d781f2ce77234d86a7596ee564f41f89]) successfully deleted
W20260929 10:50:32.304455 28361 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.80.193:34221 (error 108)
I20260929 10:50:32.304649 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 28588
I20260929 10:50:32.306460 28975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:32.306167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:34221 (local address 127.0.0.1:32964)
0929 10:50:32.306335 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:32.306407 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.193:34221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":80,"threads_started":1}
W20260929 10:50:32.306602 28361 catalog_manager.cc:4584] TS afa84438b9224029840033fd30444c3f (127.20.80.193:34221): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4d151670899841f89b99768f3e6a042c: Network error: Client connection negotiation failed: client connection to 127.20.80.193:34221: connect: Connection refused (error 111)
I20260929 10:50:32.309674 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 28729
I20260929 10:50:32.315419 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 28342
2026-09-29T10:50:32Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2656 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:50:32 dist-test-slave-vqch krb5kdc[28978](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:50:32 dist-test-slave-vqch krb5kdc[28978](info): set up 2 sockets
Sep 29 10:50:32 dist-test-slave-vqch krb5kdc[28978](info): commencing operation
krb5kdc: starting...
W20260929 10:50:33.353260 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:50:33 dist-test-slave-vqch krb5kdc[28978](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679033, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:50:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:50:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.80.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
I20260929 10:50:33.474159 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:35987
--webserver_interface=127.20.80.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:46521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:35987
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:33.554754 28991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:33.554972 28991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:33.554996 28991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:33.556102 28991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:50:33.556139 28991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:33.556154 28991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:33.556166 28991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:50:33.556178 28991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:50:33.557621 28991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:46521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:35987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:35987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:33.557812 28991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:33.557983 28991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:33.562832 28997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:33.562948 28999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:33.562875 28996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:33.563037 28991 server_base.cc:1034] running on GCE node
I20260929 10:50:33.563274 28991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:33.563512 28991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:33.564651 28991 hybrid_clock.cc:648] HybridClock initialized: now 1790679033564631 us; error 39 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-29T10:50:33Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1252 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:50:33 dist-test-slave-vqch krb5kdc[29004](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:50:33 dist-test-slave-vqch krb5kdc[29004](info): set up 2 sockets
Sep 29 10:50:33 dist-test-slave-vqch krb5kdc[29004](info): commencing operation
krb5kdc: starting...
W20260929 10:50:34.601210 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:50:34 dist-test-slave-vqch krb5kdc[29004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:50:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:50:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.80.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
I20260929 10:50:34.720736 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:37871
--webserver_interface=127.20.80.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:34867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:37871
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:34.802006 29017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:34.802240 29017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:34.802277 29017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:34.803442 29017 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:50:34.803514 29017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:34.803539 29017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:34.803570 29017 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:50:34.803596 29017 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:50:34.805132 29017 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:34867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:37871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:37871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:34.805408 29017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:34.805648 29017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:34.810856 29025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:34.810834 29023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:34.810972 29017 server_base.cc:1034] running on GCE node
W20260929 10:50:34.810818 29022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:34.811297 29017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:34.811625 29017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:34.812813 29017 hybrid_clock.cc:648] HybridClock initialized: now 1790679034812787 us; error 41 us; skew 500 ppm
Sep 29 10:50:34 dist-test-slave-vqch krb5kdc[29004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679034, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:34.819957 29017 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:50:34.820391 29017 webserver.cc:460] Webserver started at http://127.20.80.254:33065/ using document root <none> and password file <none>
I20260929 10:50:34.820600 29017 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:34.820640 29017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:34.820734 29017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:34.821898 29017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "1a895331f5964a41ba034e61a5b25923"
format_stamp: "Formatted at 2026-09-29 10:50:34 on dist-test-slave-vqch"
server_key: "5041aa418fbdde9a953ad1ab6844767c"
server_key_iv: "4da91df8c3af0c09e8d48316e99ccd9f"
server_key_version: "encryptionkey@0"
I20260929 10:50:34.822201 29017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "1a895331f5964a41ba034e61a5b25923"
format_stamp: "Formatted at 2026-09-29 10:50:34 on dist-test-slave-vqch"
server_key: "5041aa418fbdde9a953ad1ab6844767c"
server_key_iv: "4da91df8c3af0c09e8d48316e99ccd9f"
server_key_version: "encryptionkey@0"
I20260929 10:50:34.823493 29017 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 10:50:34.824252 29032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:34.824491 29017 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:50:34.824623 29017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "1a895331f5964a41ba034e61a5b25923"
format_stamp: "Formatted at 2026-09-29 10:50:34 on dist-test-slave-vqch"
server_key: "5041aa418fbdde9a953ad1ab6844767c"
server_key_iv: "4da91df8c3af0c09e8d48316e99ccd9f"
server_key_version: "encryptionkey@0"
I20260929 10:50:34.824775 29017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:34.856544 29017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:34.858700 29017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:34.858894 29017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:34.863348 29017 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:37871
I20260929 10:50:34.863394 29084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:37871 every 8 connection(s)
I20260929 10:50:34.863832 29017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:50:34.864709 29085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:34.865783 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 29017
I20260929 10:50:34.865878 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:50:34.866107 20803 external_mini_cluster.cc:1428] Setting key 7a6b806ba597f4b0bf10fb81426e5c56
I20260929 10:50:34.867381 29085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923: Bootstrap starting.
I20260929 10:50:34.868149 29085 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:34.868472 29085 log.cc:824] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:34.869222 29085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923: No bootstrap required, opened a new log
I20260929 10:50:34.871601 29085 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a895331f5964a41ba034e61a5b25923" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37871 } }
I20260929 10:50:34.871783 29085 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:34.871821 29085 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a895331f5964a41ba034e61a5b25923, State: Initialized, Role: FOLLOWER
I20260929 10:50:34.871969 29085 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a895331f5964a41ba034e61a5b25923" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37871 } }
I20260929 10:50:34.872061 29085 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:50:34.872097 29085 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:50:34.872151 29085 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:34.872845 29085 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a895331f5964a41ba034e61a5b25923" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37871 } }
I20260929 10:50:34.872973 29085 leader_election.cc:302] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a895331f5964a41ba034e61a5b25923; no voters:
Sep 29 10:50:34 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:34.873195 29085 leader_election.cc:288] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:50:34.873219 29090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:34.873339 29090 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [term 1 LEADER]: Becoming Leader. State: Replica: 1a895331f5964a41ba034e61a5b25923, State: Running, Role: LEADER
I20260929 10:50:34.873426 29090 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a895331f5964a41ba034e61a5b25923" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37871 } }
I20260929 10:50:34.873450 29085 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:50:34.873839 29092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a895331f5964a41ba034e61a5b25923. Latest consensus state: current_term: 1 leader_uuid: "1a895331f5964a41ba034e61a5b25923" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a895331f5964a41ba034e61a5b25923" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37871 } } }
I20260929 10:50:34.873924 29092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:34.874151 29094 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:50:34.874794 29094 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:50:34.875263 29091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a895331f5964a41ba034e61a5b25923" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a895331f5964a41ba034e61a5b25923" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37871 } } }
I20260929 10:50:34.875429 29091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:34.876798 29094 catalog_manager.cc:1347] Generated new cluster ID: fa4f0290f43745f390f77a9fea45b470
I20260929 10:50:34.876850 29094 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:50:34.877810 29088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:34.866662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35002 (local address 127.20.80.254:37871)
0929 10:50:34.866845 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:34.866853 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:34.866879 (+ 26us) server_negotiation.cc:407] Connection header received
0929 10:50:34.867289 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:34.867295 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:34.867339 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:34.867485 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:34.868285 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:34.869386 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:34.870397 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:34.870838 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:34.870926 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:34.873759 (+ 2833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:34.873763 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:34.873764 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:34.873776 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:34.876128 (+ 2352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:34.876550 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:34.876554 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:34.876554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:34.876666 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:34.876909 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:34.876910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:34.876910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:34.877203 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:34.877372 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:34.877477 (+ 105us) server_negotiation.cc:299] Negotiation successful
0929 10:50:34.877616 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":92,"threads_started":1}
I20260929 10:50:34.902474 29094 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:50:34.903216 29094 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:50:34.908089 29094 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923: Generated new TSK 0
I20260929 10:50:34.908264 29094 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.80.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
I20260929 10:50:34.940724 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.193:0
--local_ip_for_outbound_sockets=127.20.80.193
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37871
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:34867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:35.034307 29111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:35.034503 29111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:35.034526 29111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:35.035713 29111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:35.035754 29111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:35.035813 29111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.193
I20260929 10:50:35.037341 29111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:34867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.193
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:35.037528 29111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:35.037705 29111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:35.042635 29117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:35.042661 29111 server_base.cc:1034] running on GCE node
W20260929 10:50:35.042658 29120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:35.042632 29116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:35.043058 29111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:35.043251 29111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:35.044389 29111 hybrid_clock.cc:648] HybridClock initialized: now 1790679035044377 us; error 29 us; skew 500 ppm
Sep 29 10:50:35 dist-test-slave-vqch krb5kdc[29004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679035, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:35.046901 29111 init.cc:377] Logged in from keytab as kudu/127.20.80.193@KRBTEST.COM (short username kudu)
I20260929 10:50:35.047315 29111 webserver.cc:460] Webserver started at http://127.20.80.193:36629/ using document root <none> and password file <none>
I20260929 10:50:35.047559 29111 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:35.047605 29111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:35.047767 29111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:35.048779 29111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-0/data/instance:
uuid: "e1bd61a967ca4468877985af12d10d4a"
format_stamp: "Formatted at 2026-09-29 10:50:35 on dist-test-slave-vqch"
server_key: "0b3de39d2cacf3046eb1a7a530eb6b47"
server_key_iv: "4131a122b8cb081a01a7f63be43f9600"
server_key_version: "encryptionkey@0"
I20260929 10:50:35.049147 29111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance:
uuid: "e1bd61a967ca4468877985af12d10d4a"
format_stamp: "Formatted at 2026-09-29 10:50:35 on dist-test-slave-vqch"
server_key: "0b3de39d2cacf3046eb1a7a530eb6b47"
server_key_iv: "4131a122b8cb081a01a7f63be43f9600"
server_key_version: "encryptionkey@0"
I20260929 10:50:35.050493 29111 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:50:35.051262 29127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:35.051620 29111 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:50:35.051731 29111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-0/wal
uuid: "e1bd61a967ca4468877985af12d10d4a"
format_stamp: "Formatted at 2026-09-29 10:50:35 on dist-test-slave-vqch"
server_key: "0b3de39d2cacf3046eb1a7a530eb6b47"
server_key_iv: "4131a122b8cb081a01a7f63be43f9600"
server_key_version: "encryptionkey@0"
I20260929 10:50:35.051842 29111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:35.070922 29111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:35.072939 29111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:35.073127 29111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:35.073387 29111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:35.073704 29111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:35.073738 29111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:35.073760 29111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:35.073808 29111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:35.079553 29111 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.193:39645
I20260929 10:50:35.079636 29242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.193:39645 every 8 connection(s)
I20260929 10:50:35.079955 29111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
Sep 29 10:50:35 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679035, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:35.086684 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 29111
I20260929 10:50:35.086850 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance
I20260929 10:50:35.087105 20803 external_mini_cluster.cc:1428] Setting key 2117c9b70686d92e449b8d8f1ac1416d
I20260929 10:50:35.089843 29088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:35.080687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:57555 (local address 127.20.80.254:37871)
0929 10:50:35.080757 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:35.080760 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:35.081020 (+ 260us) server_negotiation.cc:407] Connection header received
0929 10:50:35.081553 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:35.081554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:35.081564 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:35.081625 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:35.082598 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.083085 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.084042 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.084237 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.084286 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:35.086439 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:35.086441 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:35.086441 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:35.086448 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:35.088366 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:35.088746 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:35.088748 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:35.088748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:35.088817 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:35.089091 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:35.089092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:35.089092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:35.089243 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:35.089318 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:35.089666 (+ 348us) server_negotiation.cc:299] Negotiation successful
0929 10:50:35.089746 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 10:50:35.090080 29245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:35.080679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.193:57555)
0929 10:50:35.080875 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:35.080911 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:35.081448 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:35.081706 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:35.081709 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:35.082324 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:35.082513 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:35.082518 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.083166 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.083167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:35.083968 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:35.083971 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.084308 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.084309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:35.084409 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:35.084945 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:35.084953 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:35.086344 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:35.088449 (+ 2105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:35.088456 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:35.088457 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:35.088674 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:35.088886 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:35.088886 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:35.088887 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:35.089029 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:35.089319 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:35.089327 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:35.089509 (+ 182us) client_negotiation.cc:769] Sending connection context
0929 10:50:35.089623 (+ 114us) client_negotiation.cc:240] Negotiation successful
0929 10:50:35.089801 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":95,"threads_started":1}
I20260929 10:50:35.090621 29243 heartbeater.cc:347] Connected to a master server at 127.20.80.254:37871
I20260929 10:50:35.090703 29243 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:35.090920 29243 heartbeater.cc:511] Master 127.20.80.254:37871 requested a full tablet report, sending...
I20260929 10:50:35.091468 29049 ts_manager.cc:194] Registered new tserver with Master: e1bd61a967ca4468877985af12d10d4a (127.20.80.193:39645)
I20260929 10:50:35.092140 29049 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.193@KRBTEST.COM'} at 127.20.80.193:57555
WARNING: no policy specified for kudu/127.20.80.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
I20260929 10:50:35.111698 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.194:0
--local_ip_for_outbound_sockets=127.20.80.194
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37871
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:34867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:35.192958 29248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:35.193169 29248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:35.193192 29248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:35.194397 29248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:35.194440 29248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:35.194490 29248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.194
I20260929 10:50:35.195916 29248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:34867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.194
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:35.196153 29248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:35.196379 29248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:35.201457 29256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:35.201434 29253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:35.201481 29254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:35.201474 29248 server_base.cc:1034] running on GCE node
I20260929 10:50:35.201906 29248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:35.202163 29248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:35.203318 29248 hybrid_clock.cc:648] HybridClock initialized: now 1790679035203313 us; error 35 us; skew 500 ppm
Sep 29 10:50:35 dist-test-slave-vqch krb5kdc[29004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679035, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:35.205880 29248 init.cc:377] Logged in from keytab as kudu/127.20.80.194@KRBTEST.COM (short username kudu)
I20260929 10:50:35.206311 29248 webserver.cc:460] Webserver started at http://127.20.80.194:38061/ using document root <none> and password file <none>
I20260929 10:50:35.206534 29248 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:35.206580 29248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:35.206698 29248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:35.207751 29248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-1/data/instance:
uuid: "0a986784904e41db9c4f8024b04c3e0d"
format_stamp: "Formatted at 2026-09-29 10:50:35 on dist-test-slave-vqch"
server_key: "04233d715debb7f26c04ccbcb21824ec"
server_key_iv: "3eeac57374e0c8c45fe35898da37c4d3"
server_key_version: "encryptionkey@0"
I20260929 10:50:35.208109 29248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance:
uuid: "0a986784904e41db9c4f8024b04c3e0d"
format_stamp: "Formatted at 2026-09-29 10:50:35 on dist-test-slave-vqch"
server_key: "04233d715debb7f26c04ccbcb21824ec"
server_key_iv: "3eeac57374e0c8c45fe35898da37c4d3"
server_key_version: "encryptionkey@0"
I20260929 10:50:35.209506 29248 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 10:50:35.210301 29263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:35.210495 29248 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 10:50:35.210596 29248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-1/wal
uuid: "0a986784904e41db9c4f8024b04c3e0d"
format_stamp: "Formatted at 2026-09-29 10:50:35 on dist-test-slave-vqch"
server_key: "04233d715debb7f26c04ccbcb21824ec"
server_key_iv: "3eeac57374e0c8c45fe35898da37c4d3"
server_key_version: "encryptionkey@0"
I20260929 10:50:35.210670 29248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:35.222914 29248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:35.225234 29248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:35.225389 29248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:35.225607 29248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:35.225982 29248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:35.226048 29248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:35.226091 29248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:35.226125 29248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:35.232189 29248 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.194:37231
I20260929 10:50:35.232232 29377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.194:37231 every 8 connection(s)
I20260929 10:50:35.232553 29248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
I20260929 10:50:35.236749 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 29248
I20260929 10:50:35.236871 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance
I20260929 10:50:35.237113 20803 external_mini_cluster.cc:1428] Setting key 2e09175b77c19dd8462ee69698320ec6
Sep 29 10:50:35 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679035, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:35.242276 29088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:35.233269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:49051 (local address 127.20.80.254:37871)
0929 10:50:35.233337 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:35.233339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:35.233556 (+ 217us) server_negotiation.cc:407] Connection header received
0929 10:50:35.234074 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:35.234075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:35.234087 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:35.234143 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:35.235068 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.235582 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.236561 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.236781 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.236874 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:35.239352 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:35.239353 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:35.239354 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:35.239363 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:35.240788 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:35.241308 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:35.241309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:35.241309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:35.241363 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:35.241621 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:35.241622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:35.241623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:35.241744 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:35.241821 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:35.242102 (+ 281us) server_negotiation.cc:299] Negotiation successful
0929 10:50:35.242185 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 10:50:35.242480 29380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:35.233253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.194:49051)
0929 10:50:35.233435 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:35.233470 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:35.233968 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:35.234218 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:35.234222 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:35.234784 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:35.234969 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:35.234973 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.235676 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.235676 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:35.236469 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:35.236472 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.236908 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.236909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:35.237094 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:35.237564 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:35.237572 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:35.239247 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:35.240873 (+ 1626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:35.240880 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:35.240882 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:35.241233 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:35.241437 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:35.241438 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:35.241438 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:35.241549 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:35.241812 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:35.241818 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:35.241963 (+ 145us) client_negotiation.cc:769] Sending connection context
0929 10:50:35.242067 (+ 104us) client_negotiation.cc:240] Negotiation successful
0929 10:50:35.242225 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":90,"threads_started":1}
I20260929 10:50:35.242975 29378 heartbeater.cc:347] Connected to a master server at 127.20.80.254:37871
I20260929 10:50:35.243079 29378 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:35.243251 29378 heartbeater.cc:511] Master 127.20.80.254:37871 requested a full tablet report, sending...
I20260929 10:50:35.243672 29049 ts_manager.cc:194] Registered new tserver with Master: 0a986784904e41db9c4f8024b04c3e0d (127.20.80.194:37231)
I20260929 10:50:35.244056 29049 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.194@KRBTEST.COM'} at 127.20.80.194:49051
WARNING: no policy specified for kudu/127.20.80.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
I20260929 10:50:35.263382 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.195:0
--local_ip_for_outbound_sockets=127.20.80.195
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37871
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:34867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:35.345925 29383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:35.346130 29383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:35.346153 29383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:35.347275 29383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:35.347312 29383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:35.347337 29383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.195
I20260929 10:50:35.348800 29383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:34867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.195
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:35.349053 29383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:35.349262 29383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:35.354369 29388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:35.354360 29389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:35.354346 29391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:35.354547 29383 server_base.cc:1034] running on GCE node
I20260929 10:50:35.354830 29383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:35.355096 29383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:35.356281 29383 hybrid_clock.cc:648] HybridClock initialized: now 1790679035356270 us; error 38 us; skew 500 ppm
Sep 29 10:50:35 dist-test-slave-vqch krb5kdc[29004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679035, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:35.358989 29383 init.cc:377] Logged in from keytab as kudu/127.20.80.195@KRBTEST.COM (short username kudu)
I20260929 10:50:35.359465 29383 webserver.cc:460] Webserver started at http://127.20.80.195:38935/ using document root <none> and password file <none>
I20260929 10:50:35.359716 29383 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:35.359787 29383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:35.359934 29383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:35.361047 29383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-2/data/instance:
uuid: "2b2240d2381946de800a28c3875ca682"
format_stamp: "Formatted at 2026-09-29 10:50:35 on dist-test-slave-vqch"
server_key: "dafc5b2e0e58c182701e41983d5b4e17"
server_key_iv: "acea8251d2a7c231763bac02a63c6a8d"
server_key_version: "encryptionkey@0"
I20260929 10:50:35.361426 29383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance:
uuid: "2b2240d2381946de800a28c3875ca682"
format_stamp: "Formatted at 2026-09-29 10:50:35 on dist-test-slave-vqch"
server_key: "dafc5b2e0e58c182701e41983d5b4e17"
server_key_iv: "acea8251d2a7c231763bac02a63c6a8d"
server_key_version: "encryptionkey@0"
I20260929 10:50:35.362733 29383 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:35.363440 29398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:35.363611 29383 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:35.363704 29383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-2/wal
uuid: "2b2240d2381946de800a28c3875ca682"
format_stamp: "Formatted at 2026-09-29 10:50:35 on dist-test-slave-vqch"
server_key: "dafc5b2e0e58c182701e41983d5b4e17"
server_key_iv: "acea8251d2a7c231763bac02a63c6a8d"
server_key_version: "encryptionkey@0"
I20260929 10:50:35.363785 29383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:35.376652 29383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:35.379065 29383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:35.379231 29383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:35.379449 29383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:35.379760 29383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:35.379794 29383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:35.379817 29383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:35.379860 29383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:35.385330 29383 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.195:43067
I20260929 10:50:35.385397 29512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.195:43067 every 8 connection(s)
I20260929 10:50:35.385733 29383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
I20260929 10:50:35.388309 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 29383
I20260929 10:50:35.388401 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance
I20260929 10:50:35.388581 20803 external_mini_cluster.cc:1428] Setting key f0d671042472eba85a346bb21771643d
Sep 29 10:50:35 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679035, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:35.395071 29088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:35.386455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:51507 (local address 127.20.80.254:37871)
0929 10:50:35.386534 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:35.386536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:35.386727 (+ 191us) server_negotiation.cc:407] Connection header received
0929 10:50:35.387175 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:35.387176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:35.387186 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:35.387244 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:35.388180 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.388917 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.390129 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.390337 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.390394 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:35.392522 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:35.392523 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:35.392523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:35.392530 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:35.393972 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:35.394317 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:35.394318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:35.394318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:35.394365 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:35.394565 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:35.394566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:35.394566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:35.394661 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:35.394710 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:35.394949 (+ 239us) server_negotiation.cc:299] Negotiation successful
0929 10:50:35.395007 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 10:50:35.395287 29515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:35.386404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.195:51507)
0929 10:50:35.386584 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:35.386619 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:35.387091 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:35.387312 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:35.387315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:35.387865 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:35.388056 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:35.388060 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.389016 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.389017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:35.390052 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:35.390055 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.390415 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.390416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:35.390515 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:35.390999 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:35.391006 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:35.392435 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:35.394039 (+ 1604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:35.394044 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:35.394046 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:35.394242 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:35.394422 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:35.394423 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:35.394423 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:35.394513 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:35.394714 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:35.394718 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:35.394826 (+ 108us) client_negotiation.cc:769] Sending connection context
0929 10:50:35.394923 (+ 97us) client_negotiation.cc:240] Negotiation successful
0929 10:50:35.395039 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":97,"threads_started":1}
I20260929 10:50:35.395826 29513 heartbeater.cc:347] Connected to a master server at 127.20.80.254:37871
I20260929 10:50:35.395938 29513 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:35.396122 29513 heartbeater.cc:511] Master 127.20.80.254:37871 requested a full tablet report, sending...
I20260929 10:50:35.396520 29049 ts_manager.cc:194] Registered new tserver with Master: 2b2240d2381946de800a28c3875ca682 (127.20.80.195:43067)
I20260929 10:50:35.396890 29049 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.195@KRBTEST.COM'} at 127.20.80.195:51507
I20260929 10:50:35.400249 20803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.20.80.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.80.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.80.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.193.keytab.
Entry for principal oryx/127.20.80.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.193.keytab.
WARNING: no policy specified for oryx/127.20.80.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.80.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.80.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.194.keytab.
Entry for principal oryx/127.20.80.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.194.keytab.
WARNING: no policy specified for oryx/127.20.80.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.80.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.80.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.195.keytab.
Entry for principal oryx/127.20.80.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.195.keytab.
I20260929 10:50:35.471900 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 29111
I20260929 10:50:35.477118 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 29248
I20260929 10:50:35.481312 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 29383
I20260929 10:50:35.485483 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 29017
I20260929 10:50:35.490940 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:37871
--webserver_interface=127.20.80.254
--webserver_port=33065
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:34867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:37871
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:35.571810 29522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:35.572070 29522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:35.572110 29522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:35.573313 29522 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:50:35.573385 29522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:35.573410 29522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:35.573439 29522 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:50:35.573463 29522 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:50:35.574939 29522 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:34867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:37871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:37871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=33065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:35.575220 29522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:35.575459 29522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:35.580235 29527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:35.580235 29530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:35.580282 29528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:35.580582 29522 server_base.cc:1034] running on GCE node
I20260929 10:50:35.580816 29522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:35.581138 29522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:35.582326 29522 hybrid_clock.cc:648] HybridClock initialized: now 1790679035582308 us; error 45 us; skew 500 ppm
Sep 29 10:50:35 dist-test-slave-vqch krb5kdc[29004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679035, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:35.584888 29522 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:50:35.585387 29522 webserver.cc:460] Webserver started at http://127.20.80.254:33065/ using document root <none> and password file <none>
I20260929 10:50:35.585630 29522 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:35.585716 29522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:35.587023 29522 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:35.587644 29537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:35.587813 29522 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:35.587918 29522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "1a895331f5964a41ba034e61a5b25923"
format_stamp: "Formatted at 2026-09-29 10:50:34 on dist-test-slave-vqch"
server_key: "5041aa418fbdde9a953ad1ab6844767c"
server_key_iv: "4da91df8c3af0c09e8d48316e99ccd9f"
server_key_version: "encryptionkey@0"
I20260929 10:50:35.588258 29522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:35.610476 29522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:35.612807 29522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:35.613044 29522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:35.617287 29522 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:37871
I20260929 10:50:35.617321 29590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:37871 every 8 connection(s)
I20260929 10:50:35.617728 29522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:50:35.619199 29591 sys_catalog.cc:263] Verifying existing consensus state
I20260929 10:50:35.619853 29591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923: Bootstrap starting.
I20260929 10:50:35.625829 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 29522
I20260929 10:50:35.626170 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.193:39645
--local_ip_for_outbound_sockets=127.20.80.193
--tserver_master_addrs=127.20.80.254:37871
--webserver_port=36629
--webserver_interface=127.20.80.193
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:34867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.20.80.193
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:50:35.628758 29591 log.cc:824] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:35.630529 29591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 10:50:35.630808 29591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923: Bootstrap complete.
I20260929 10:50:35.633272 29591 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a895331f5964a41ba034e61a5b25923" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37871 } }
I20260929 10:50:35.633498 29591 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a895331f5964a41ba034e61a5b25923, State: Initialized, Role: FOLLOWER
I20260929 10:50:35.633656 29591 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a895331f5964a41ba034e61a5b25923" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37871 } }
I20260929 10:50:35.633723 29591 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:50:35.633783 29591 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:50:35.633831 29591 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [term 1 FOLLOWER]: Advancing to term 2
I20260929 10:50:35.634825 29591 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a895331f5964a41ba034e61a5b25923" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37871 } }
I20260929 10:50:35.634913 29591 leader_election.cc:302] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a895331f5964a41ba034e61a5b25923; no voters:
I20260929 10:50:35.635119 29591 leader_election.cc:288] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 10:50:35.635154 29595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 10:50:35.635270 29595 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [term 2 LEADER]: Becoming Leader. State: Replica: 1a895331f5964a41ba034e61a5b25923, State: Running, Role: LEADER
I20260929 10:50:35.635352 29595 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a895331f5964a41ba034e61a5b25923" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37871 } }
I20260929 10:50:35.635394 29591 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:50:35.635610 29595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1a895331f5964a41ba034e61a5b25923" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a895331f5964a41ba034e61a5b25923" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37871 } } }
I20260929 10:50:35.635675 29595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:35.635782 29595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a895331f5964a41ba034e61a5b25923. Latest consensus state: current_term: 2 leader_uuid: "1a895331f5964a41ba034e61a5b25923" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a895331f5964a41ba034e61a5b25923" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37871 } } }
I20260929 10:50:35.635848 29595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923 [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:35.636003 29599 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:50:35.636513 29599 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:50:35.636655 29599 catalog_manager.cc:1259] Loaded cluster ID: fa4f0290f43745f390f77a9fea45b470
I20260929 10:50:35.639938 29599 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:50:35.641301 29599 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:50:35.641505 29599 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1a895331f5964a41ba034e61a5b25923: Loaded TSK: 0
I20260929 10:50:35.641733 29599 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 10:50:35.709339 29593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:35.709560 29593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:35.709599 29593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:35.710753 29593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:35.710819 29593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:35.710865 29593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.193
I20260929 10:50:35.712306 29593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:34867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.193.keytab
--principal=oryx/127.20.80.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.193:39645
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.80.193
--webserver_port=36629
--tserver_master_addrs=127.20.80.254:37871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.193
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:35.712584 29593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:35.712802 29593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:35.718137 29618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:35.718124 29617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:35.718112 29620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:35.718578 29593 server_base.cc:1034] running on GCE node
I20260929 10:50:35.718791 29593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:35.719048 29593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:35.720204 29593 hybrid_clock.cc:648] HybridClock initialized: now 1790679035720186 us; error 33 us; skew 500 ppm
Sep 29 10:50:35 dist-test-slave-vqch krb5kdc[29004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679035, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:35.722730 29593 init.cc:377] Logged in from keytab as oryx/127.20.80.193@KRBTEST.COM (short username oryx)
I20260929 10:50:35.723224 29593 webserver.cc:460] Webserver started at http://127.20.80.193:36629/ using document root <none> and password file <none>
I20260929 10:50:35.723465 29593 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:35.723520 29593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:35.724928 29593 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:35.725643 29627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:35.725831 29593 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 10:50:35.725929 29593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-0/wal
uuid: "e1bd61a967ca4468877985af12d10d4a"
format_stamp: "Formatted at 2026-09-29 10:50:35 on dist-test-slave-vqch"
server_key: "0b3de39d2cacf3046eb1a7a530eb6b47"
server_key_iv: "4131a122b8cb081a01a7f63be43f9600"
server_key_version: "encryptionkey@0"
I20260929 10:50:35.726291 29593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:35.739683 29593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:35.742015 29593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:35.742177 29593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:35.742441 29593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:35.742816 29593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:35.742870 29593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:35.742911 29593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:35.742937 29593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:35.750101 29593 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.193:39645
I20260929 10:50:35.750172 29741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.193:39645 every 8 connection(s)
I20260929 10:50:35.750535 29593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
I20260929 10:50:35.751618 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 29593
I20260929 10:50:35.751973 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.194:37231
--local_ip_for_outbound_sockets=127.20.80.194
--tserver_master_addrs=127.20.80.254:37871
--webserver_port=38061
--webserver_interface=127.20.80.194
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:34867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.20.80.194
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 10:50:35 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:35 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:35.763757 29744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:35.751260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.193:43463)
0929 10:50:35.753493 (+ 2233us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:35.753543 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:35.754171 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:35.754810 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:35.754815 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:35.755647 (+ 832us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:35.756007 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:35.756015 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.757654 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.757656 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:35.759137 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:35.759140 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.759907 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.759909 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:35.760066 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:35.760707 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:35.760718 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:35.763551 (+ 2833us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":2185,"thread_start_us":93,"threads_started":1}
W20260929 10:50:35.763823 29745 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:35.751383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:43463 (local address 127.20.80.254:37871)
0929 10:50:35.754379 (+ 2996us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:35.754388 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:35.754422 (+ 34us) server_negotiation.cc:407] Connection header received
0929 10:50:35.754540 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:35.754546 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:35.754597 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:35.754720 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:35.756145 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.757543 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.759239 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.759800 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.759916 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:35.763672 (+ 3756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.193:43463: BlockingRecv error: recv got EOF from 127.20.80.193:43463 (error 108)
Metrics: {"server-negotiator.queue_time_us":2933,"thread_start_us":74,"threads_started":1}
W20260929 10:50:35.764006 29742 heartbeater.cc:644] Failed to heartbeat to 127.20.80.254:37871 (0 consecutive failures): Not authorized: Failed to ping master at 127.20.80.254:37871: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Sep 29 10:50:35 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:35 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:35.768970 29744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:35.764239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.193:39959)
0929 10:50:35.764299 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:35.764312 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:35.764376 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:35.764513 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:35.764514 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:35.764628 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:35.764673 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:35.764674 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.765642 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.765643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:35.766140 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:35.766142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.766592 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.766593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:35.766641 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:35.767099 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:35.767104 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:35.768892 (+ 1788us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260929 10:50:35.769114 29745 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:35.764308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:39959 (local address 127.20.80.254:37871)
0929 10:50:35.764366 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:35.764368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:35.764378 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:50:35.764434 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:35.764435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:35.764447 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:35.764500 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:35.764735 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.765552 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.766210 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.766521 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.766586 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:35.769028 (+ 2442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.193:39959: BlockingRecv error: recv got EOF from 127.20.80.193:39959 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 29 10:50:35 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:35 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:35.773782 29744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:35.769249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.193:39451)
0929 10:50:35.769294 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:35.769306 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:35.769344 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:35.769504 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:35.769505 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:35.769657 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:35.769698 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:35.769699 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.770308 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.770309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:35.771121 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:35.771123 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.771438 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.771439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:35.771492 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:35.771892 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:35.771898 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:35.773687 (+ 1789us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260929 10:50:35.773813 29745 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:35.769306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:39451 (local address 127.20.80.254:37871)
0929 10:50:35.769338 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:35.769340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:35.769348 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:50:35.769417 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:35.769418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:35.769426 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:35.769476 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:35.769774 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.770236 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.771198 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.771371 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.771421 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:35.773754 (+ 2333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.193:39451: BlockingRecv error: recv got EOF from 127.20.80.193:39451 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260929 10:50:35.773933 29742 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 10:50:35.837971 29746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:35.838178 29746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:35.838202 29746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:35.839309 29746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:35.839346 29746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:35.839372 29746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.194
I20260929 10:50:35.840709 29746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:34867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.194.keytab
--principal=oryx/127.20.80.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.194:37231
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.80.194
--webserver_port=38061
--tserver_master_addrs=127.20.80.254:37871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.194
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:35.840898 29746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:35.841099 29746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:35.846029 29754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:35.846113 29751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:35.846016 29752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:35.846205 29746 server_base.cc:1034] running on GCE node
I20260929 10:50:35.846439 29746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:35.846686 29746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:35.847863 29746 hybrid_clock.cc:648] HybridClock initialized: now 1790679035847841 us; error 32 us; skew 500 ppm
Sep 29 10:50:35 dist-test-slave-vqch krb5kdc[29004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679035, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:35.850380 29746 init.cc:377] Logged in from keytab as oryx/127.20.80.194@KRBTEST.COM (short username oryx)
I20260929 10:50:35.850870 29746 webserver.cc:460] Webserver started at http://127.20.80.194:38061/ using document root <none> and password file <none>
I20260929 10:50:35.851100 29746 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:35.851183 29746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:35.852607 29746 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:35.853333 29761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:35.853506 29746 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:50:35.853615 29746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-1/wal
uuid: "0a986784904e41db9c4f8024b04c3e0d"
format_stamp: "Formatted at 2026-09-29 10:50:35 on dist-test-slave-vqch"
server_key: "04233d715debb7f26c04ccbcb21824ec"
server_key_iv: "3eeac57374e0c8c45fe35898da37c4d3"
server_key_version: "encryptionkey@0"
I20260929 10:50:35.853982 29746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:35.888679 29746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:35.891088 29746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:35.891279 29746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:35.891595 29746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:35.891939 29746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:35.891976 29746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:35.892025 29746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:35.892066 29746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:35.898353 29746 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.194:37231
I20260929 10:50:35.898399 29874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.194:37231 every 8 connection(s)
I20260929 10:50:35.898761 29746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
Sep 29 10:50:35 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:35 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
W20260929 10:50:35.906131 29745 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:35.899515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:40413 (local address 127.20.80.254:37871)
0929 10:50:35.899560 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:35.899562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:35.899770 (+ 208us) server_negotiation.cc:407] Connection header received
0929 10:50:35.900324 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:35.900326 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:35.900339 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:35.900391 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:35.901456 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.901954 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.902973 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.903152 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.903200 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:35.906082 (+ 2882us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.194:40413: BlockingRecv error: recv got EOF from 127.20.80.194:40413 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 10:50:35.906149 29877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:35.899500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.194:40413)
0929 10:50:35.899658 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:35.899695 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:35.900185 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:35.900458 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:35.900461 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:35.901146 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:35.901365 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:35.901371 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.902034 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.902035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:35.902895 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:35.902897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.903247 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.903248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:35.903358 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:35.903808 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:35.903816 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:35.906003 (+ 2187us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":89,"threads_started":1}
W20260929 10:50:35.906378 29875 heartbeater.cc:644] Failed to heartbeat to 127.20.80.254:37871 (0 consecutive failures): Not authorized: Failed to ping master at 127.20.80.254:37871: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
I20260929 10:50:35.907537 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 29746
I20260929 10:50:35.907823 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.195:43067
--local_ip_for_outbound_sockets=127.20.80.195
--tserver_master_addrs=127.20.80.254:37871
--webserver_port=38935
--webserver_interface=127.20.80.195
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:34867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.20.80.195
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 10:50:35 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:35 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:35.911345 29877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:35.906628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.194:54107)
0929 10:50:35.906683 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:35.906694 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:35.906737 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:35.906889 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:35.906890 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:35.906993 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:35.907033 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:35.907035 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.907672 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.907673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:35.908320 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:35.908323 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.908783 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.908784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:35.908869 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:35.909323 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:35.909328 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:35.911242 (+ 1914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260929 10:50:35.911446 29745 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:35.906692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:54107 (local address 127.20.80.254:37871)
0929 10:50:35.906773 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:35.906775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:35.906783 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:50:35.906803 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:35.906804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:35.906812 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:35.906863 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:35.907094 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.907580 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.908429 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.908640 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.908781 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:35.911356 (+ 2575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.194:54107: BlockingRecv error: recv got EOF from 127.20.80.194:54107 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 29 10:50:35 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:35 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:35.916456 29877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:35.911735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.194:34009)
0929 10:50:35.911799 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:35.911813 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:35.911883 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:35.912045 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:35.912046 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:35.912181 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:35.912209 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:35.912210 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.912933 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.912934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:35.913511 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:35.913512 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.913924 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.913925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:35.913981 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:35.914469 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:35.914474 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:35.916373 (+ 1899us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":41}
W20260929 10:50:35.916512 29745 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:35.911834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:34009 (local address 127.20.80.254:37871)
0929 10:50:35.911891 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:35.911893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:35.911921 (+ 28us) server_negotiation.cc:407] Connection header received
0929 10:50:35.911943 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:35.911944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:35.911960 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:35.912040 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:35.912316 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.912845 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.913619 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:35.913844 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:35.913926 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:35.916443 (+ 2517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.194:34009: BlockingRecv error: recv got EOF from 127.20.80.194:34009 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260929 10:50:35.916746 29875 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 10:50:35.991315 29878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:35.991549 29878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:35.991581 29878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:35.992884 29878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:35.992944 29878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:35.993007 29878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.195
I20260929 10:50:35.994414 29878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:34867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/krb5kdc/oryx_127.20.80.195.keytab
--principal=oryx/127.20.80.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.195:43067
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.80.195
--webserver_port=38935
--tserver_master_addrs=127.20.80.254:37871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.195
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:35.994592 29878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:35.994756 29878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:35.999756 29886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:35.999826 29884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:35.999990 29878 server_base.cc:1034] running on GCE node
W20260929 10:50:35.999975 29883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:36.000306 29878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:36.000577 29878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:36.001752 29878 hybrid_clock.cc:648] HybridClock initialized: now 1790679036001729 us; error 34 us; skew 500 ppm
Sep 29 10:50:36 dist-test-slave-vqch krb5kdc[29004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679036, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.80.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:36.004190 29878 init.cc:377] Logged in from keytab as oryx/127.20.80.195@KRBTEST.COM (short username oryx)
I20260929 10:50:36.004614 29878 webserver.cc:460] Webserver started at http://127.20.80.195:38935/ using document root <none> and password file <none>
I20260929 10:50:36.004864 29878 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:36.004956 29878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:36.006319 29878 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:50:36.006897 29893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:36.007054 29878 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:50:36.007164 29878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-2/wal
uuid: "2b2240d2381946de800a28c3875ca682"
format_stamp: "Formatted at 2026-09-29 10:50:35 on dist-test-slave-vqch"
server_key: "dafc5b2e0e58c182701e41983d5b4e17"
server_key_iv: "acea8251d2a7c231763bac02a63c6a8d"
server_key_version: "encryptionkey@0"
I20260929 10:50:36.007501 29878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:36.021654 29878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:36.023939 29878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:36.024102 29878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:36.024338 29878 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:36.024648 29878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:36.024710 29878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:36.024751 29878 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:36.024781 29878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:36.030751 29878 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.195:43067
I20260929 10:50:36.030800 30006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.195:43067 every 8 connection(s)
I20260929 10:50:36.031126 29878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
I20260929 10:50:36.032842 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 29878
Sep 29 10:50:36 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:36 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
W20260929 10:50:36.038887 29745 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:36.031797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:41197 (local address 127.20.80.254:37871)
0929 10:50:36.031850 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:36.031853 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:36.032072 (+ 219us) server_negotiation.cc:407] Connection header received
0929 10:50:36.032551 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:36.032552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:36.032563 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:36.032615 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:36.033843 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:36.034316 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:36.035394 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:36.035592 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:36.035668 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:36.038828 (+ 3160us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.195:41197: BlockingRecv error: recv got EOF from 127.20.80.195:41197 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 10:50:36.038973 30010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:36.031791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.195:41197)
0929 10:50:36.031952 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:36.031987 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:36.032463 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:36.032708 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:36.032712 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:36.033464 (+ 752us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:36.033761 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:36.033768 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:36.034416 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:36.034417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:36.035291 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:36.035293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:36.035676 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:36.035677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:36.035821 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:36.036374 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:36.036384 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:36.038770 (+ 2386us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":86,"threads_started":1}
W20260929 10:50:36.039172 30007 heartbeater.cc:644] Failed to heartbeat to 127.20.80.254:37871 (0 consecutive failures): Not authorized: Failed to ping master at 127.20.80.254:37871: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
I20260929 10:50:36.040566 30012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:36.033428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35014 (local address 127.20.80.254:37871)
0929 10:50:36.033555 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:36.033557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:36.033569 (+ 12us) server_negotiation.cc:407] Connection header received
0929 10:50:36.033593 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:36.033594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:36.033602 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:36.033657 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:36.034299 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:36.034826 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:36.035628 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:36.035883 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:36.035954 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:36.037272 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:36.037276 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:36.037277 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:36.037286 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:36.039003 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:36.039496 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:36.039498 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:36.039498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:36.039582 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:36.039808 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:36.039809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:36.039809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:36.040040 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:36.040183 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:36.040352 (+ 169us) server_negotiation.cc:299] Negotiation successful
0929 10:50:36.040465 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":86,"threads_started":1}
Sep 29 10:50:36 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:36 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:36.043642 30010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:36.039384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.195:37853)
0929 10:50:36.039430 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:36.039440 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:36.039476 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:36.039603 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:36.039604 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:36.039731 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:36.039770 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:36.039771 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:36.040385 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:36.040386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:36.041201 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:36.041203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:36.041659 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:36.041660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:36.041710 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:36.042024 (+ 314us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:36.042028 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:36.043600 (+ 1572us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260929 10:50:36.043815 29745 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:36.039443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:37853 (local address 127.20.80.254:37871)
0929 10:50:36.039481 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:36.039483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:36.039508 (+ 25us) server_negotiation.cc:407] Connection header received
0929 10:50:36.039527 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:36.039527 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:36.039535 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:36.039603 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:36.039842 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:36.040290 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:36.041312 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:36.041594 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:36.041645 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:36.043702 (+ 2057us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.195:37853: BlockingRecv error: recv got EOF from 127.20.80.195:37853 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Sep 29 10:50:36 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:36 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:36.047937 30010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:36.044081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.195:37881)
0929 10:50:36.044131 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:36.044141 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:36.044184 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:36.044295 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:36.044296 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:36.044414 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:36.044440 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:36.044441 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:36.045087 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:36.045088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:36.045579 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:36.045581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:36.045954 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:36.045955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:36.046007 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:36.046403 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:36.046408 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:36.047862 (+ 1454us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260929 10:50:36.047986 29745 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:36.044137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:37881 (local address 127.20.80.254:37871)
0929 10:50:36.044193 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:36.044195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:36.044206 (+ 11us) server_negotiation.cc:407] Connection header received
0929 10:50:36.044228 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:36.044228 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:36.044237 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:36.044282 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:36.044524 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:36.044967 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:36.045677 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:36.045869 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:36.045952 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:36.047924 (+ 1972us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.195:37881: BlockingRecv error: recv got EOF from 127.20.80.195:37881 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260929 10:50:36.048144 30007 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 29 10:50:36 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:36 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:36.779172 30013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:36.774307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.193:49585)
0929 10:50:36.774465 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:36.774478 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:36.774539 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:36.774677 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:36.774678 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:36.774864 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:36.774919 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:36.774921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:36.775640 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:36.775641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:36.776293 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:36.776294 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:36.776687 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:36.776688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:36.776768 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:36.777223 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:36.777228 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:36.779103 (+ 1875us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":86,"threads_started":1}
W20260929 10:50:36.779247 30014 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:36.774428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:49585 (local address 127.20.80.254:37871)
0929 10:50:36.774531 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:36.774533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:36.774549 (+ 16us) server_negotiation.cc:407] Connection header received
0929 10:50:36.774589 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:36.774589 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:36.774599 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:36.774654 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:36.775024 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:36.775535 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:36.776385 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:36.776580 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:36.776637 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:36.779193 (+ 2556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.193:49585: BlockingRecv error: recv got EOF from 127.20.80.193:49585 (error 108)
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":55,"threads_started":1}
Sep 29 10:50:36 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:36 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:36.921947 30015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:36.917180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.194:36239)
0929 10:50:36.917327 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:36.917342 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:36.917435 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:36.917578 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:36.917578 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:36.917767 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:36.917815 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:36.917817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:36.918491 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:36.918492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:36.919043 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:36.919044 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:36.919412 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:36.919413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:36.919472 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:36.920042 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:36.920047 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:36.921876 (+ 1829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":76,"threads_started":1}
W20260929 10:50:36.922127 30014 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:36.917266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:36239 (local address 127.20.80.254:37871)
0929 10:50:36.917342 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:36.917345 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:36.917416 (+ 71us) server_negotiation.cc:407] Connection header received
0929 10:50:36.917506 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:36.917506 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:36.917520 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:36.917572 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:36.917906 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:36.918406 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:36.919133 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:36.919337 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:36.919413 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:36.922032 (+ 2619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.194:36239: BlockingRecv error: recv got EOF from 127.20.80.194:36239 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 29 10:50:37 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:37 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:37.053277 30016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:37.048540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.195:51161)
0929 10:50:37.048743 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:37.048753 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:37.048828 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:37.049001 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:37.049002 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:37.049173 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:37.049221 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:37.049222 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:37.049886 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:37.049887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:37.050462 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:37.050464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:37.050821 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:37.050822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:37.050885 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:37.051299 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:37.051304 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:37.053209 (+ 1905us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":85,"threads_started":1}
W20260929 10:50:37.053388 30014 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:37.048592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:51161 (local address 127.20.80.254:37871)
0929 10:50:37.048661 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:37.048664 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:37.048818 (+ 154us) server_negotiation.cc:407] Connection header received
0929 10:50:37.048885 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:37.048886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:37.048902 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:37.048975 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:37.049300 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:37.049797 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:37.050548 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:37.050746 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:37.050824 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:37.053310 (+ 2486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.195:51161: BlockingRecv error: recv got EOF from 127.20.80.195:51161 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 29 10:50:37 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:37 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:37.784291 30017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:37.779720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.193:60245)
0929 10:50:37.779862 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:37.779872 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:37.779922 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:37.780046 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:37.780046 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:37.780220 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:37.780279 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:37.780281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:37.780941 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:37.780942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:37.781502 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:37.781504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:37.781833 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:37.781834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:37.781878 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:37.782346 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:37.782351 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:37.784231 (+ 1880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":80,"threads_started":1}
W20260929 10:50:37.784433 30018 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:37.779800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:60245 (local address 127.20.80.254:37871)
0929 10:50:37.779914 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:37.779916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:37.779932 (+ 16us) server_negotiation.cc:407] Connection header received
0929 10:50:37.779957 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:37.779958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:37.779967 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:37.780026 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:37.780350 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:37.780843 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:37.781569 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:37.781757 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:37.781809 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:37.784337 (+ 2528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.193:60245: BlockingRecv error: recv got EOF from 127.20.80.193:60245 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":71,"threads_started":1}
Sep 29 10:50:37 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:37 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:37.927232 30019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:37.922408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.194:60569)
0929 10:50:37.922579 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:37.922591 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:37.922678 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:37.922834 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:37.922835 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:37.923021 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:37.923068 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:37.923070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:37.923759 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:37.923760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:37.924300 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:37.924301 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:37.924664 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:37.924665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:37.924722 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:37.925282 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:37.925287 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:37.927156 (+ 1869us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":93,"threads_started":1}
W20260929 10:50:37.927484 30018 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:37.922498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:60569 (local address 127.20.80.254:37871)
0929 10:50:37.922571 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:37.922573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:37.922658 (+ 85us) server_negotiation.cc:407] Connection header received
0929 10:50:37.922743 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:37.922744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:37.922756 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:37.922829 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:37.923148 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:37.923669 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:37.924385 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:37.924589 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:37.924665 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:37.927401 (+ 2736us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.194:60569: BlockingRecv error: recv got EOF from 127.20.80.194:60569 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 29 10:50:38 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:38 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:38.058771 30020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:38.053778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.195:50061)
0929 10:50:38.053944 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:38.053956 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:38.054023 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:38.054196 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:38.054197 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:38.054392 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:38.054439 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:38.054440 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:38.055119 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:38.055120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:38.055665 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:38.055667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:38.056057 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:38.056058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:38.056118 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:38.056640 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:38.056645 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:38.058694 (+ 2049us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":68,"threads_started":1}
W20260929 10:50:38.058822 30018 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:38.053904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:50061 (local address 127.20.80.254:37871)
0929 10:50:38.053967 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:38.053970 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:38.054002 (+ 32us) server_negotiation.cc:407] Connection header received
0929 10:50:38.054098 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:38.054099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:38.054114 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:38.054191 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:38.054528 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:38.055037 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:38.055750 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:38.055949 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:38.056047 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:38.058776 (+ 2729us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.195:50061: BlockingRecv error: recv got EOF from 127.20.80.195:50061 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 29 10:50:38 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:38 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:38.789275 30021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:38.784853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.193:34555)
0929 10:50:38.785133 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:38.785143 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:38.785194 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:38.785339 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:38.785340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:38.785493 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:38.785542 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:38.785543 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:38.786179 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:38.786180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:38.786701 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:38.786703 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:38.787053 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:38.787054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:38.787094 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:38.787501 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:38.787505 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:38.789216 (+ 1711us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":204,"threads_started":1}
W20260929 10:50:38.789456 30022 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:38.785053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:34555 (local address 127.20.80.254:37871)
0929 10:50:38.785202 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:38.785205 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:38.785213 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:50:38.785235 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:38.785235 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:38.785253 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:38.785317 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:38.785601 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:38.786112 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:38.786775 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:38.786984 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:38.787039 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:38.789322 (+ 2283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.193:34555: BlockingRecv error: recv got EOF from 127.20.80.193:34555 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":96,"threads_started":1}
Sep 29 10:50:38 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:38 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:38.932502 30023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:38.927681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.194:56353)
0929 10:50:38.927866 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:38.927879 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:38.927972 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:38.928134 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:38.928135 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:38.928415 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:38.928464 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:38.928466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:38.929224 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:38.929225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:38.929800 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:38.929801 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:38.930164 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:38.930165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:38.930225 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:38.930706 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:38.930711 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:38.932393 (+ 1682us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":105,"threads_started":1}
W20260929 10:50:38.932549 30022 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:38.927780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:56353 (local address 127.20.80.254:37871)
0929 10:50:38.927880 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:38.927882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:38.927950 (+ 68us) server_negotiation.cc:407] Connection header received
0929 10:50:38.928044 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:38.928045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:38.928058 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:38.928130 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:38.928547 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:38.929146 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:38.929882 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:38.930090 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:38.930165 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:38.932485 (+ 2320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.194:56353: BlockingRecv error: recv got EOF from 127.20.80.194:56353 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 29 10:50:39 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:39 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:39.063920 30024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:39.059246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.195:58045)
0929 10:50:39.059446 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:39.059458 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:39.059558 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:39.059749 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:39.059750 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:39.059935 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:39.059983 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:39.059985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:39.060651 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:39.060653 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:39.061226 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:39.061228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:39.061605 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:39.061606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:39.061674 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:39.062099 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:39.062104 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:39.063826 (+ 1722us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":101,"threads_started":1}
W20260929 10:50:39.063980 30022 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:39.059329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:58045 (local address 127.20.80.254:37871)
0929 10:50:39.059443 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:39.059446 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:39.059539 (+ 93us) server_negotiation.cc:407] Connection header received
0929 10:50:39.059631 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:39.059632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:39.059659 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:39.059739 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:39.060065 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:39.060565 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:39.061318 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:39.061517 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:39.061597 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:39.063918 (+ 2321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.195:58045: BlockingRecv error: recv got EOF from 127.20.80.195:58045 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Sep 29 10:50:39 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:39 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:39.794588 30025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:39.789750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.193:33269)
0929 10:50:39.789912 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:39.789922 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:39.789978 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:39.790134 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:39.790135 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:39.790300 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:39.790352 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:39.790354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:39.791115 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:39.791116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:39.791835 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:39.791840 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:39.792334 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:39.792335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:39.792379 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:39.792808 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:39.792812 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:39.794529 (+ 1717us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":90,"threads_started":1}
W20260929 10:50:39.794792 30026 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:39.789835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:33269 (local address 127.20.80.254:37871)
0929 10:50:39.789946 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:39.789948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:39.789961 (+ 13us) server_negotiation.cc:407] Connection header received
0929 10:50:39.790040 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:39.790040 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:39.790051 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:39.790109 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:39.790482 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:39.791044 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:39.791920 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:39.792262 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:39.792324 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:39.794701 (+ 2377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.193:33269: BlockingRecv error: recv got EOF from 127.20.80.193:33269 (error 108)
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":50,"threads_started":1}
Sep 29 10:50:39 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:39 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:39.944576 30027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:39.933040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.194:37389)
0929 10:50:39.933157 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:39.933175 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:39.933233 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:39.933345 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:39.933346 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:39.933499 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:39.933546 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:39.933548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:39.934418 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:39.934420 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:39.935418 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:39.935420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:39.936000 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:39.936001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:39.936084 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:39.942446 (+ 6362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:39.942451 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:39.944516 (+ 2065us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":70,"threads_started":1}
W20260929 10:50:39.944720 30026 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:39.933072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:37389 (local address 127.20.80.254:37871)
0929 10:50:39.933115 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:39.933117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:39.933233 (+ 116us) server_negotiation.cc:407] Connection header received
0929 10:50:39.933275 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:39.933276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:39.933294 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:39.933342 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:39.933601 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:39.934300 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:39.935516 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:39.935907 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:39.935983 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:39.944640 (+ 8657us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.194:37389: BlockingRecv error: recv got EOF from 127.20.80.194:37389 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
Sep 29 10:50:40 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:40 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:40.069339 30028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:40.064422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.195:59623)
0929 10:50:40.064611 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:40.064623 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:40.064695 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:40.064876 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:40.064878 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:40.065096 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:40.065150 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:40.065152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:40.065815 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:40.065816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:40.066366 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:40.066368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:40.066719 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:40.066720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:40.066781 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:40.067293 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:40.067298 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:40.069219 (+ 1921us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":90,"threads_started":1}
W20260929 10:50:40.069378 30026 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:40.064516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:59623 (local address 127.20.80.254:37871)
0929 10:50:40.064595 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:40.064597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:40.064675 (+ 78us) server_negotiation.cc:407] Connection header received
0929 10:50:40.064786 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:40.064787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:40.064801 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:40.064867 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:40.065232 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:40.065724 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:40.066453 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:40.066644 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:40.066720 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:40.069313 (+ 2593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.195:59623: BlockingRecv error: recv got EOF from 127.20.80.195:59623 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 29 10:50:40 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:40 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:40.799760 30029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:40.795053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.193:47877)
0929 10:50:40.795243 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:40.795255 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:40.795318 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:40.795461 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:40.795462 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:40.795637 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:40.795691 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:40.795693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:40.796347 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:40.796348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:40.796943 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:40.796944 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:40.797370 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:40.797371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:40.797420 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:40.797867 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:40.797872 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:40.799663 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":84,"threads_started":1}
W20260929 10:50:40.799819 30030 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:40.795148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:47877 (local address 127.20.80.254:37871)
0929 10:50:40.795273 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:40.795275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:40.795299 (+ 24us) server_negotiation.cc:407] Connection header received
0929 10:50:40.795371 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:40.795372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:40.795382 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:40.795437 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:40.795753 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:40.796266 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:40.797106 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:40.797299 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:40.797352 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:40.799744 (+ 2392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.193:47877: BlockingRecv error: recv got EOF from 127.20.80.193:47877 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":71,"threads_started":1}
Sep 29 10:50:40 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:40 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:40.950112 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:40.945193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.194:60379)
0929 10:50:40.945343 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:40.945357 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:40.945426 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:40.945661 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:40.945662 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:40.945924 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:40.945987 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:40.945990 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:40.946861 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:40.946862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:40.947420 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:40.947422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:40.947756 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:40.947757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:40.947805 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:40.948273 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:40.948278 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:40.950053 (+ 1775us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":90,"threads_started":1}
W20260929 10:50:40.950232 30030 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:40.945339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:60379 (local address 127.20.80.254:37871)
0929 10:50:40.945405 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:40.945408 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:40.945419 (+ 11us) server_negotiation.cc:407] Connection header received
0929 10:50:40.945544 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:40.945545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:40.945560 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:40.945650 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:40.946077 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:40.946751 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:40.947501 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:40.947688 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:40.947743 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:40.950153 (+ 2410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.194:60379: BlockingRecv error: recv got EOF from 127.20.80.194:60379 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Sep 29 10:50:41 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:41 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:41.075187 30032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:41.069978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.195:46341)
0929 10:50:41.070145 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:41.070156 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:41.070212 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:41.070383 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:41.070384 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:41.070752 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:41.070805 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:41.070807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:41.071591 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:41.071592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:41.072178 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:41.072180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:41.072579 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:41.072580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:41.072628 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:41.073252 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:41.073257 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:41.075088 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":104,"threads_started":1}
W20260929 10:50:41.075235 30030 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:41.070045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:46341 (local address 127.20.80.254:37871)
0929 10:50:41.070129 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:41.070131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:41.070216 (+ 85us) server_negotiation.cc:407] Connection header received
0929 10:50:41.070271 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:41.070271 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:41.070282 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:41.070325 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:41.070927 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:41.071482 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:41.072276 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:41.072496 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:41.072563 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:41.075172 (+ 2609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.195:46341: BlockingRecv error: recv got EOF from 127.20.80.195:46341 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 29 10:50:41 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:41 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:41.805135 30033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:41.800300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.193:53845)
0929 10:50:41.800450 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:41.800466 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:41.800516 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:41.800693 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:41.800694 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:41.800849 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:41.800907 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:41.800909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:41.801661 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:41.801662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:41.802278 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:41.802280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:41.802677 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:41.802678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:41.802743 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:41.803231 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:41.803242 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:41.805075 (+ 1833us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":87,"threads_started":1}
W20260929 10:50:41.805298 30034 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:41.800401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:53845 (local address 127.20.80.254:37871)
0929 10:50:41.800534 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:41.800536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:41.800552 (+ 16us) server_negotiation.cc:407] Connection header received
0929 10:50:41.800575 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:41.800576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:41.800587 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:41.800651 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:41.801054 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:41.801583 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:41.802396 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:41.802593 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:41.802676 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:41.805188 (+ 2512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.193:53845: BlockingRecv error: recv got EOF from 127.20.80.193:53845 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":87,"threads_started":1}
Sep 29 10:50:41 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:41 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:41.955567 30035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:41.950619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.194:44063)
0929 10:50:41.950790 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:41.950804 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:41.950902 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:41.951044 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:41.951045 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:41.951235 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:41.951282 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:41.951284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:41.951971 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:41.951972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:41.952511 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:41.952512 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:41.952886 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:41.952887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:41.952945 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:41.953465 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:41.953470 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:41.955469 (+ 1999us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":93,"threads_started":1}
W20260929 10:50:41.955619 30034 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:41.950716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:44063 (local address 127.20.80.254:37871)
0929 10:50:41.950795 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:41.950797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:41.950883 (+ 86us) server_negotiation.cc:407] Connection header received
0929 10:50:41.950971 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:41.950972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:41.950983 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:41.951041 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:41.951373 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:41.951878 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:41.952598 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:41.952809 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:41.952888 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:41.955553 (+ 2665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.194:44063: BlockingRecv error: recv got EOF from 127.20.80.194:44063 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 29 10:50:42 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:42 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:42.080241 30037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:42.075710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.195:58297)
0929 10:50:42.075828 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:42.075840 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:42.075907 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:42.076049 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:42.076050 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:42.076209 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:42.076257 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:42.076259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:42.076918 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:42.076918 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:42.077489 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:42.077491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:42.077854 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:42.077855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:42.077895 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:42.078372 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:42.078376 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:42.080141 (+ 1765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":56,"threads_started":1}
W20260929 10:50:42.080288 30034 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:42.075787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:58297 (local address 127.20.80.254:37871)
0929 10:50:42.075874 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:42.075875 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:42.075888 (+ 13us) server_negotiation.cc:407] Connection header received
0929 10:50:42.075966 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:42.075967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:42.075979 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:42.076029 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:42.076329 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:42.076849 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:42.077569 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:42.077788 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:42.077845 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:42.080216 (+ 2371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.195:58297: BlockingRecv error: recv got EOF from 127.20.80.195:58297 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 29 10:50:42 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:42 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:42.810333 30038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:42.805579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.193:45615)
0929 10:50:42.805716 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:42.805726 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:42.805775 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:42.805989 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:42.805989 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:42.806143 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:42.806195 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:42.806196 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:42.806851 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:42.806852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:42.807376 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:42.807377 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:42.807771 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:42.807772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:42.807819 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:42.808287 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:42.808291 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:42.810271 (+ 1980us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
W20260929 10:50:42.810504 30039 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:42.805727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:45615 (local address 127.20.80.254:37871)
0929 10:50:42.805850 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:42.805852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:42.805861 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:50:42.805892 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:42.805893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:42.805904 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:42.805970 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:42.806272 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:42.806783 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:42.807488 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:42.807690 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:42.807759 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:42.810428 (+ 2669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.193:45615: BlockingRecv error: recv got EOF from 127.20.80.193:45615 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":72,"threads_started":1}
Sep 29 10:50:42 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:42 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:42.960726 30040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:42.956115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.194:50417)
0929 10:50:42.956269 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:42.956279 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:42.956329 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:42.956529 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:42.956530 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:42.956738 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:42.956788 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:42.956790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:42.957455 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:42.957456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:42.958045 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:42.958048 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:42.958370 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:42.958371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:42.958411 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:42.958834 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:42.958839 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:42.960668 (+ 1829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
W20260929 10:50:42.960875 30039 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:42.956265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:50417 (local address 127.20.80.254:37871)
0929 10:50:42.956325 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:42.956327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:42.956367 (+ 40us) server_negotiation.cc:407] Connection header received
0929 10:50:42.956393 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:42.956394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:42.956405 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:42.956460 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:42.956876 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:42.957387 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:42.958118 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:42.958305 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:42.958357 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:42.960796 (+ 2439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.194:50417: BlockingRecv error: recv got EOF from 127.20.80.194:50417 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 29 10:50:43 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:43 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:43.085642 30041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:43.080849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.195:53031)
0929 10:50:43.081052 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:43.081067 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:43.081126 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:43.081272 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:43.081273 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:43.081436 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:43.081484 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:43.081485 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:43.082189 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:43.082190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:43.082722 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:43.082724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:43.083068 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:43.083069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:43.083113 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:43.083593 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:43.083597 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:43.085542 (+ 1945us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":98,"threads_started":1}
W20260929 10:50:43.085891 30039 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:43.080950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:53031 (local address 127.20.80.254:37871)
0929 10:50:43.081015 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:43.081018 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:43.081128 (+ 110us) server_negotiation.cc:407] Connection header received
0929 10:50:43.081191 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:43.081192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:43.081205 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:43.081249 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:43.081571 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:43.082090 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:43.082801 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:43.082991 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:43.083043 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:43.085631 (+ 2588us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.195:53031: BlockingRecv error: recv got EOF from 127.20.80.195:53031 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 29 10:50:43 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:43 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:43.815979 30042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:43.810943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.193:42833)
0929 10:50:43.811124 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:43.811138 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:43.811208 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:43.811343 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:43.811343 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:43.811510 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:43.811560 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:43.811562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:43.812303 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:43.812304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:43.812847 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:43.812848 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:43.813276 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:43.813277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:43.813339 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:43.813906 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:43.813911 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:43.815871 (+ 1960us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":61,"threads_started":1}
W20260929 10:50:43.816023 30043 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:43.811072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:42833 (local address 127.20.80.254:37871)
0929 10:50:43.811207 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:43.811209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:43.811219 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:50:43.811249 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:43.811249 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:43.811262 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:43.811328 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:43.811641 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:43.812229 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:43.812943 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:43.813166 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:43.813236 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:43.815948 (+ 2712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.193:42833: BlockingRecv error: recv got EOF from 127.20.80.193:42833 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":62,"threads_started":1}
Sep 29 10:50:43 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:43 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:43.966434 30044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:43.961322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.194:54031)
0929 10:50:43.961501 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:43.961514 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:43.961616 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:43.961837 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:43.961838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:43.962028 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:43.962075 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:43.962078 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:43.962928 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:43.962929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:43.963487 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:43.963488 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:43.963879 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:43.963880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:43.963946 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:43.964390 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:43.964394 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:43.966342 (+ 1948us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":97,"threads_started":1}
W20260929 10:50:43.966499 30043 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:43.961408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:54031 (local address 127.20.80.254:37871)
0929 10:50:43.961493 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:43.961495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:43.961594 (+ 99us) server_negotiation.cc:407] Connection header received
0929 10:50:43.961704 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:43.961706 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:43.961757 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:43.961826 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:43.962162 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:43.962834 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:43.963579 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:43.963789 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:43.963877 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:43.966436 (+ 2559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.194:54031: BlockingRecv error: recv got EOF from 127.20.80.194:54031 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 29 10:50:44 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:44 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:44.090891 30045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:44.086160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.195:48677)
0929 10:50:44.086300 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:44.086312 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:44.086387 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:44.086543 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:44.086544 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:44.086730 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:44.086776 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:44.086778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:44.087464 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:44.087465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:44.087994 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:44.087995 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:44.088386 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:44.088387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:44.088452 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:44.088940 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:44.088945 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:44.090783 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":65,"threads_started":1}
W20260929 10:50:44.090937 30043 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:44.086205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:48677 (local address 127.20.80.254:37871)
0929 10:50:44.086260 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:44.086262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:44.086364 (+ 102us) server_negotiation.cc:407] Connection header received
0929 10:50:44.086451 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:44.086452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:44.086465 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:44.086535 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:44.086882 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:44.087380 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:44.088081 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:44.088275 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:44.088365 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:44.090870 (+ 2505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.195:48677: BlockingRecv error: recv got EOF from 127.20.80.195:48677 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Sep 29 10:50:44 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:44 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:44.821563 30046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:44.816509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.193:39427)
0929 10:50:44.816711 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:44.816723 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:44.816801 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:44.817010 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:44.817011 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:44.817204 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:44.817263 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:44.817265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:44.818013 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:44.818014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:44.818557 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:44.818559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:44.818996 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:44.818997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:44.819062 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:44.819521 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:44.819526 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:44.821463 (+ 1937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":85,"threads_started":1}
W20260929 10:50:44.821626 30047 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:44.816633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:39427 (local address 127.20.80.254:37871)
0929 10:50:44.816752 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:44.816754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:44.816779 (+ 25us) server_negotiation.cc:407] Connection header received
0929 10:50:44.816880 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:44.816881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:44.816892 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:44.816950 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:44.817368 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:44.817910 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:44.818666 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:44.818881 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:44.818980 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:44.821553 (+ 2573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.193:39427: BlockingRecv error: recv got EOF from 127.20.80.193:39427 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":62,"threads_started":1}
Sep 29 10:50:44 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:44 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:44.975003 30048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:44.967002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.194:43021)
0929 10:50:44.967188 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:44.967198 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:44.967258 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:44.967406 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:44.967407 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:44.967572 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:44.967623 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:44.967625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:44.968268 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:44.968269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:44.968802 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:44.968803 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:44.969283 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:44.969285 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:44.969347 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:44.973011 (+ 3664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:44.973017 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:44.974934 (+ 1917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":99,"threads_started":1}
W20260929 10:50:44.975239 30047 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:44.967096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:43021 (local address 127.20.80.254:37871)
0929 10:50:44.967165 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:44.967166 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:44.967261 (+ 95us) server_negotiation.cc:407] Connection header received
0929 10:50:44.967303 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:44.967304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:44.967314 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:44.967376 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:44.967691 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:44.968195 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:44.968906 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:44.969186 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:44.969250 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:44.975132 (+ 5882us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.194:43021: BlockingRecv error: recv got EOF from 127.20.80.194:43021 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 29 10:50:45 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:45 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:45.096338 30049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:45.091399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.195:32805)
0929 10:50:45.091556 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:45.091575 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:45.091669 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:45.091882 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:45.091883 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:45.092064 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:45.092112 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:45.092113 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:45.092823 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:45.092824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:45.093400 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:45.093402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:45.093793 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:45.093794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:45.093859 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:45.094379 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:45.094383 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:45.096278 (+ 1895us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":81,"threads_started":1}
W20260929 10:50:45.096464 30047 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:45.091524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:32805 (local address 127.20.80.254:37871)
0929 10:50:45.091571 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:45.091573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:45.091647 (+ 74us) server_negotiation.cc:407] Connection header received
0929 10:50:45.091785 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:45.091787 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:45.091809 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:45.091875 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:45.092204 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:45.092722 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:45.093505 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:45.093704 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:45.093793 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:45.096370 (+ 2577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.195:32805: BlockingRecv error: recv got EOF from 127.20.80.195:32805 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Sep 29 10:50:45 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:45 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:45.826947 30050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:45.822073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.193:60523)
0929 10:50:45.822275 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:45.822289 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:45.822356 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:45.822524 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:45.822525 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:45.822692 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:45.822742 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:45.822743 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:45.823616 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:45.823617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:45.824203 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:45.824205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:45.824558 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:45.824559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:45.824598 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:45.825100 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:45.825105 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:45.826878 (+ 1773us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":73,"threads_started":1}
W20260929 10:50:45.827054 30051 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:45.822140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:60523 (local address 127.20.80.254:37871)
0929 10:50:45.822372 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:45.822374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:45.822386 (+ 12us) server_negotiation.cc:407] Connection header received
0929 10:50:45.822422 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:45.822423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:45.822435 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:45.822506 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:45.822817 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:45.823522 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:45.824296 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:45.824490 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:45.824547 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:45.826978 (+ 2431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.193:60523: BlockingRecv error: recv got EOF from 127.20.80.193:60523 (error 108)
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":148,"threads_started":1}
Sep 29 10:50:45 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:45 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:45.980016 30052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:45.975578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.194:37123)
0929 10:50:45.975720 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:45.975730 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:45.975782 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:45.975912 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:45.975913 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:45.976076 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:45.976124 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:45.976126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:45.976770 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:45.976771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:45.977338 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:45.977340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:45.977682 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:45.977683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:45.977734 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:45.978182 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:45.978186 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:45.979957 (+ 1771us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":52,"threads_started":1}
W20260929 10:50:45.980099 30051 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:45.975632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:37123 (local address 127.20.80.254:37871)
0929 10:50:45.975680 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:45.975682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:45.975779 (+ 97us) server_negotiation.cc:407] Connection header received
0929 10:50:45.975836 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:45.975836 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:45.975847 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:45.975890 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:45.976188 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:45.976684 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:45.977412 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:45.977595 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:45.977649 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:45.980050 (+ 2401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.194:37123: BlockingRecv error: recv got EOF from 127.20.80.194:37123 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Sep 29 10:50:46 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:46 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:46.101143 30053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:46.096824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.195:49891)
0929 10:50:46.096972 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:46.097020 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:46.097079 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:46.097211 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:46.097211 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:46.097374 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:46.097423 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:46.097425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:46.098041 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:46.098041 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:46.098551 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:46.098552 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:46.098853 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:46.098854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:46.098893 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:46.099327 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:46.099331 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:46.101075 (+ 1744us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":68,"threads_started":1}
W20260929 10:50:46.101228 30051 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:46.096948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:49891 (local address 127.20.80.254:37871)
0929 10:50:46.097019 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:46.097021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:46.097064 (+ 43us) server_negotiation.cc:407] Connection header received
0929 10:50:46.097134 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:46.097135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:46.097148 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:46.097187 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:46.097490 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:46.097975 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:46.098619 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:46.098790 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:46.098838 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:46.101162 (+ 2324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.195:49891: BlockingRecv error: recv got EOF from 127.20.80.195:49891 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 29 10:50:46 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:46 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:46.832331 30054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:46.827525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.193:49761)
0929 10:50:46.827681 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:46.827692 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:46.827742 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:46.827937 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:46.827938 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:46.828111 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:46.828159 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:46.828160 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:46.828913 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:46.828914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:46.829523 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:46.829525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:46.829958 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:46.829959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:46.830010 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:46.830555 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:46.830560 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:46.832264 (+ 1704us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":99,"threads_started":1}
W20260929 10:50:46.832494 30055 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:46.827592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:49761 (local address 127.20.80.254:37871)
0929 10:50:46.827812 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:46.827814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:46.827822 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:50:46.827845 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:46.827845 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:46.827856 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:46.827922 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:46.828225 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:46.828837 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:46.829651 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:46.829893 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:46.829950 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:46.832384 (+ 2434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.193:49761: BlockingRecv error: recv got EOF from 127.20.80.193:49761 (error 108)
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":130,"threads_started":1}
Sep 29 10:50:46 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:46 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:46.985340 30056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:46.980675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.194:36419)
0929 10:50:46.980844 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:46.980853 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:46.980913 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:46.981072 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:46.981073 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:46.981243 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:46.981292 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:46.981294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:46.982001 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:46.982002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:46.982559 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:46.982560 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:46.982929 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:46.982930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:46.982972 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:46.983424 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:46.983428 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:46.985271 (+ 1843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":96,"threads_started":1}
W20260929 10:50:46.985477 30055 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:46.980753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:36419 (local address 127.20.80.254:37871)
0929 10:50:46.980838 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:46.980840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:46.980915 (+ 75us) server_negotiation.cc:407] Connection header received
0929 10:50:46.980991 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:46.980992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:46.981005 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:46.981050 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:46.981390 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:46.981913 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:46.982649 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:46.982860 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:46.982921 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:46.985374 (+ 2453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.194:36419: BlockingRecv error: recv got EOF from 127.20.80.194:36419 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 29 10:50:47 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:47 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:47.106319 30057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:47.101614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.195:39367)
0929 10:50:47.101811 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:47.101822 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:47.101911 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:47.102053 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:47.102054 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:47.102236 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:47.102280 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:47.102281 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:47.102967 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:47.102968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:47.103524 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:47.103526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:47.103912 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:47.103913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:47.104001 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:47.104434 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:47.104438 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:47.106208 (+ 1770us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":111,"threads_started":1}
W20260929 10:50:47.106360 30055 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:47.101697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:39367 (local address 127.20.80.254:37871)
0929 10:50:47.101789 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:47.101791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:47.101909 (+ 118us) server_negotiation.cc:407] Connection header received
0929 10:50:47.101980 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:47.101981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:47.101996 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:47.102037 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:47.102358 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:47.102872 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:47.103612 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:47.103818 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:47.103912 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:47.106294 (+ 2382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.195:39367: BlockingRecv error: recv got EOF from 127.20.80.195:39367 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 29 10:50:47 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:47 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:47.837677 30058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:47.832837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.193:60383)
0929 10:50:47.833012 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:47.833030 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:47.833080 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:47.833221 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:47.833221 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:47.833409 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:47.833459 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:47.833461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:47.834155 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:47.834156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:47.834711 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:47.834713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:47.835044 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:47.835044 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:47.835089 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:47.835797 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:47.835803 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:47.837618 (+ 1815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":93,"threads_started":1}
W20260929 10:50:47.837821 30059 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:47.832941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:60383 (local address 127.20.80.254:37871)
0929 10:50:47.833076 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:47.833078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:47.833086 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:50:47.833126 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:47.833127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:47.833139 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:47.833213 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:47.833548 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:47.834070 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:47.834783 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:47.834976 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:47.835028 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:47.837728 (+ 2700us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.193:60383: BlockingRecv error: recv got EOF from 127.20.80.193:60383 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
Sep 29 10:50:47 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:47 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:47.990587 30060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:47.985831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.194:39815)
0929 10:50:47.985967 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:47.985986 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:47.986066 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:47.986240 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:47.986241 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:47.986430 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:47.986477 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:47.986480 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:47.987154 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:47.987156 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:47.987700 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:47.987701 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:47.988134 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:47.988135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:47.988202 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:47.988677 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:47.988689 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:47.990494 (+ 1805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":68,"threads_started":1}
W20260929 10:50:47.990646 30059 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:47.985895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:39815 (local address 127.20.80.254:37871)
0929 10:50:47.985952 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:47.985954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:47.986046 (+ 92us) server_negotiation.cc:407] Connection header received
0929 10:50:47.986145 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:47.986146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:47.986160 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:47.986232 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:47.986560 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:47.987070 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:47.987841 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:47.988044 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:47.988134 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:47.990578 (+ 2444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.194:39815: BlockingRecv error: recv got EOF from 127.20.80.194:39815 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Sep 29 10:50:48 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:48 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
W20260929 10:50:48.111696 30059 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:48.106955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:37499 (local address 127.20.80.254:37871)
0929 10:50:48.107025 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:48.107027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:48.107096 (+ 69us) server_negotiation.cc:407] Connection header received
0929 10:50:48.107156 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:48.107157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:48.107169 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:48.107211 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:48.107539 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:48.108042 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:48.108743 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:48.108943 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:48.109027 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:48.111647 (+ 2620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.195:37499: BlockingRecv error: recv got EOF from 127.20.80.195:37499 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 10:50:48.111644 30061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:48.106870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.195:37499)
0929 10:50:48.107022 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:48.107040 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:48.107099 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:48.107233 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:48.107233 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:48.107424 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:48.107474 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:48.107476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:48.108112 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:48.108112 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:48.108655 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:48.108657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:48.109101 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:48.109102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:48.109161 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:48.109585 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:48.109589 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:48.111565 (+ 1976us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":99,"threads_started":1}
Sep 29 10:50:48 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:48 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:48.842895 30062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:48.838140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.193:60321)
0929 10:50:48.838296 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:48.838309 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:48.838370 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:48.838514 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:48.838514 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:48.838698 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:48.838752 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:48.838754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:48.839423 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:48.839424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:48.840022 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:48.840024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:48.840371 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:48.840372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:48.840423 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:48.840873 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:48.840877 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:48.842783 (+ 1906us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":84,"threads_started":1}
W20260929 10:50:48.842962 30063 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:48.838241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:60321 (local address 127.20.80.254:37871)
0929 10:50:48.838369 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:48.838370 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:48.838378 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:50:48.838419 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:48.838420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:48.838431 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:48.838491 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:48.838839 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:48.839347 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:48.840104 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:48.840298 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:48.840352 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:48.842888 (+ 2536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.193:60321: BlockingRecv error: recv got EOF from 127.20.80.193:60321 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":69,"threads_started":1}
Sep 29 10:50:48 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:48 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:48.995775 30064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:48.991089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.194:37277)
0929 10:50:48.991244 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:48.991255 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:48.991335 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:48.991476 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:48.991477 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:48.991650 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:48.991700 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:48.991702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:48.992544 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:48.992546 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:48.993125 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:48.993128 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:48.993454 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:48.993455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:48.993495 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:48.994004 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:48.994009 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:48.995715 (+ 1706us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":107,"threads_started":1}
W20260929 10:50:48.995930 30063 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:48.991186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:37277 (local address 127.20.80.254:37871)
0929 10:50:48.991249 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:48.991251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:48.991330 (+ 79us) server_negotiation.cc:407] Connection header received
0929 10:50:48.991381 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:48.991382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:48.991392 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:48.991444 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:48.991858 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:48.992452 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:48.993196 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:48.993388 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:48.993444 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:48.995854 (+ 2410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.194:37277: BlockingRecv error: recv got EOF from 127.20.80.194:37277 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 29 10:50:49 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:49 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:49.117439 30065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:49.112284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.195:56469)
0929 10:50:49.112432 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:49.112444 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:49.112511 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:49.112681 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:49.112682 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:49.112854 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:49.112922 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:49.112924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:49.113677 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:49.113681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:49.114262 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:49.114264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:49.114644 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:49.114645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:49.114716 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:49.115239 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:49.115244 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:49.117358 (+ 2114us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
W20260929 10:50:49.117516 30063 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:49.112344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:56469 (local address 127.20.80.254:37871)
0929 10:50:49.112405 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:49.112416 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:49.112496 (+ 80us) server_negotiation.cc:407] Connection header received
0929 10:50:49.112560 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:49.112561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:49.112574 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:49.112626 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:49.113037 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:49.113571 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:49.114357 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:49.114550 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:49.114618 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:49.117467 (+ 2849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.195:56469: BlockingRecv error: recv got EOF from 127.20.80.195:56469 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Sep 29 10:50:49 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:49 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:49.848798 30066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:49.843606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.193:46397)
0929 10:50:49.843793 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:49.843813 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:49.843892 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:49.844168 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:49.844170 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:49.844394 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:49.844446 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:49.844449 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:49.845262 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:49.845264 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:49.845803 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:49.845805 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:49.846203 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:49.846204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:49.846273 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:49.846756 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:49.846760 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:49.848702 (+ 1942us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":133,"threads_started":1}
W20260929 10:50:49.848865 30067 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:49.843774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:46397 (local address 127.20.80.254:37871)
0929 10:50:49.843919 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:49.843921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:49.843930 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:50:49.843958 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:49.843959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:49.843971 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:49.844050 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:49.844556 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:49.845170 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:49.845921 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:49.846124 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:49.846203 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:49.848796 (+ 2593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.193:46397: BlockingRecv error: recv got EOF from 127.20.80.193:46397 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":74,"threads_started":1}
Sep 29 10:50:49 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:50 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:50.006168 30068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:49.996359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.194:39079)
0929 10:50:49.996495 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:49.996507 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:49.996569 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:49.996844 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:49.996845 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:49.997104 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:49.997170 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:49.997173 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:49.997961 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:49.997962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:49.998528 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:49.998529 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:49.998925 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:49.998926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:49.998989 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:50.003883 (+ 4894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:50.003898 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:50.006047 (+ 2149us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
W20260929 10:50:50.006245 30067 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:49.996463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:39079 (local address 127.20.80.254:37871)
0929 10:50:49.996630 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:49.996632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:49.996646 (+ 14us) server_negotiation.cc:407] Connection header received
0929 10:50:49.996690 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:49.996690 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:49.996704 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:49.996813 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:49.997243 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:49.997875 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:49.998619 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:49.998836 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:49.998925 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:50.006181 (+ 7256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.194:39079: BlockingRecv error: recv got EOF from 127.20.80.194:39079 (error 108)
Metrics: {"server-negotiator.queue_time_us":137}
Sep 29 10:50:50 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:50 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.195@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:50.123097 30069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:50.118142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.195:57991)
0929 10:50:50.118343 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:50.118354 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:50.118461 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:50.118632 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:50.118633 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:50.118835 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:50.118881 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:50.118883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:50.119552 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:50.119553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:50.120105 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:50.120107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:50.120489 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:50.120490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:50.120570 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:50.121212 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:50.121218 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:50.123034 (+ 1816us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":96,"threads_started":1}
W20260929 10:50:50.123265 30067 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:50.118251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:57991 (local address 127.20.80.254:37871)
0929 10:50:50.118339 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:50.118341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:50.118463 (+ 122us) server_negotiation.cc:407] Connection header received
0929 10:50:50.118530 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:50.118531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:50.118547 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:50.118605 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:50.118965 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:50.119468 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:50.120196 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:50.120399 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:50.120487 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:50.123150 (+ 2663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.195:57991: BlockingRecv error: recv got EOF from 127.20.80.195:57991 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Sep 29 10:50:50 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:50 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.193@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:50.854578 30070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:50.849497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.193:59185)
0929 10:50:50.849653 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:50.849671 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:50.849724 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:50.849938 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:50.849940 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:50.850113 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:50.850166 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:50.850168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:50.850921 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:50.850922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:50.851719 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:50.851722 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:50.852100 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:50.852101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:50.852146 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:50.852589 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:50.852593 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:50.854480 (+ 1887us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":84,"threads_started":1}
W20260929 10:50:50.854632 30071 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:50.849591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:59185 (local address 127.20.80.254:37871)
0929 10:50:50.849724 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:50.849726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:50.849753 (+ 27us) server_negotiation.cc:407] Connection header received
0929 10:50:50.849780 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:50.849780 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:50.849791 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:50.849861 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:50.850279 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:50.850806 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:50.851807 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:50.852028 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:50.852084 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:50.854560 (+ 2476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.193:59185: BlockingRecv error: recv got EOF from 127.20.80.193:59185 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":72,"threads_started":1}
Sep 29 10:50:51 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:50:51 dist-test-slave-vqch krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.80.194@KRBTEST.COM for oryx/127.20.80.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:50:51.011559 30072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:51.006816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37871 (local address 127.20.80.194:54427)
0929 10:50:51.006953 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:51.006962 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:51.007017 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:51.007133 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:51.007133 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:51.007300 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:51.007349 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:51.007350 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:51.008063 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:51.008064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:51.008595 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:51.008596 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:51.009043 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:51.009044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:51.009097 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:51.009564 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:51.009570 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:51.011500 (+ 1930us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.80.254:37871: Server oryx/127.20.80.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":73,"threads_started":1}
W20260929 10:50:51.011709 30071 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:51.006881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:54427 (local address 127.20.80.254:37871)
0929 10:50:51.006951 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:51.006955 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:51.007002 (+ 47us) server_negotiation.cc:407] Connection header received
0929 10:50:51.007076 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:51.007077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:51.007087 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:51.007129 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:51.007449 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:51.007979 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:51.008729 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:51.008921 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:51.009014 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:51.011593 (+ 2579us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.80.194:54427: BlockingRecv error: recv got EOF from 127.20.80.194:54427 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 10:50:51.038743 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 29593
I20260929 10:50:51.043437 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 29746
I20260929 10:50:51.047772 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 29878
I20260929 10:50:51.052150 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 29522
2026-09-29T10:50:51Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17480 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-29T10:50:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:50:51Z Disabled control of system clock
I20260929 10:50:51.080720 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:44501
--webserver_interface=127.20.80.254
--webserver_port=0
--builtin_ntp_servers=127.20.80.212:44749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:44501
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260929 10:50:51.163012 30076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:51.163214 30076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:51.163242 30076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:51.164345 30076 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:50:51.164378 30076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:51.164391 30076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:51.164405 30076 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:50:51.164417 30076 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:50:51.165861 30076 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:44749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:44501
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:44501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:51.166055 30076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:51.166307 30076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:51.172792 30082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:51.172840 30084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:51.172890 30081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:51.172857 30076 server_base.cc:1034] running on GCE node
I20260929 10:50:51.173337 30076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:51.173676 30076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:51.174863 30076 hybrid_clock.cc:648] HybridClock initialized: now 1790679051174843 us; error 41 us; skew 500 ppm
I20260929 10:50:51.176258 30076 webserver.cc:460] Webserver started at http://127.20.80.254:45871/ using document root <none> and password file <none>
I20260929 10:50:51.176544 30076 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:51.176599 30076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:51.176745 30076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:51.177924 30076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "279c0ad804164f67aafe5adb93d1d676"
format_stamp: "Formatted at 2026-09-29 10:50:51 on dist-test-slave-vqch"
server_key: "f3fcd3c2993f593aef69531f39ed783f"
server_key_iv: "5234defd21ef6c0995be2ab2505cc69f"
server_key_version: "encryptionkey@0"
I20260929 10:50:51.178289 30076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "279c0ad804164f67aafe5adb93d1d676"
format_stamp: "Formatted at 2026-09-29 10:50:51 on dist-test-slave-vqch"
server_key: "f3fcd3c2993f593aef69531f39ed783f"
server_key_iv: "5234defd21ef6c0995be2ab2505cc69f"
server_key_version: "encryptionkey@0"
I20260929 10:50:51.179638 30076 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 10:50:51.180491 30090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:51.180672 30076 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:50:51.180766 30076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "279c0ad804164f67aafe5adb93d1d676"
format_stamp: "Formatted at 2026-09-29 10:50:51 on dist-test-slave-vqch"
server_key: "f3fcd3c2993f593aef69531f39ed783f"
server_key_iv: "5234defd21ef6c0995be2ab2505cc69f"
server_key_version: "encryptionkey@0"
I20260929 10:50:51.180826 30076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:51.197708 30076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:51.198010 30076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:51.198117 30076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:51.201978 30076 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:44501
I20260929 10:50:51.202013 30143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:44501 every 8 connection(s)
I20260929 10:50:51.202382 30076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:50:51.203198 30144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:51.205205 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 30076
I20260929 10:50:51.205310 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:50:51.205534 20803 external_mini_cluster.cc:1428] Setting key d9d6f9e8b3157310c543793513c75215
I20260929 10:50:51.206120 30144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 279c0ad804164f67aafe5adb93d1d676: Bootstrap starting.
I20260929 10:50:51.206748 30144 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 279c0ad804164f67aafe5adb93d1d676: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:51.207098 30144 log.cc:824] T 00000000000000000000000000000000 P 279c0ad804164f67aafe5adb93d1d676: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:51.207832 30144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 279c0ad804164f67aafe5adb93d1d676: No bootstrap required, opened a new log
I20260929 10:50:51.210070 30144 raft_consensus.cc:348] T 00000000000000000000000000000000 P 279c0ad804164f67aafe5adb93d1d676 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "279c0ad804164f67aafe5adb93d1d676" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44501 } }
I20260929 10:50:51.210263 30144 raft_consensus.cc:374] T 00000000000000000000000000000000 P 279c0ad804164f67aafe5adb93d1d676 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:51.210307 30144 raft_consensus.cc:729] T 00000000000000000000000000000000 P 279c0ad804164f67aafe5adb93d1d676 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 279c0ad804164f67aafe5adb93d1d676, State: Initialized, Role: FOLLOWER
I20260929 10:50:51.210470 30144 consensus_queue.cc:260] T 00000000000000000000000000000000 P 279c0ad804164f67aafe5adb93d1d676 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "279c0ad804164f67aafe5adb93d1d676" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44501 } }
I20260929 10:50:51.210577 30144 raft_consensus.cc:388] T 00000000000000000000000000000000 P 279c0ad804164f67aafe5adb93d1d676 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:50:51.210611 30144 raft_consensus.cc:482] T 00000000000000000000000000000000 P 279c0ad804164f67aafe5adb93d1d676 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:50:51.210657 30144 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 279c0ad804164f67aafe5adb93d1d676 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:51.211418 30144 raft_consensus.cc:504] T 00000000000000000000000000000000 P 279c0ad804164f67aafe5adb93d1d676 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "279c0ad804164f67aafe5adb93d1d676" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44501 } }
I20260929 10:50:51.211536 30144 leader_election.cc:302] T 00000000000000000000000000000000 P 279c0ad804164f67aafe5adb93d1d676 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 279c0ad804164f67aafe5adb93d1d676; no voters:
I20260929 10:50:51.211570 30147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:51.206100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38478 (local address 127.20.80.254:44501)
0929 10:50:51.206333 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:51.206341 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:51.206365 (+ 24us) server_negotiation.cc:407] Connection header received
0929 10:50:51.206830 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:51.206840 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:51.206923 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:51.207048 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:51.207371 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:51.208477 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:51.209599 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:51.209999 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:51.210104 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:51.210783 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:51.210787 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:51.210788 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:50:51.210995 (+ 207us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:51.211039 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:50:51.211054 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:51.211122 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:51.211230 (+ 108us) server_negotiation.cc:299] Negotiation successful
0929 10:50:51.211320 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":90,"threads_started":1}
I20260929 10:50:51.211714 30144 leader_election.cc:288] T 00000000000000000000000000000000 P 279c0ad804164f67aafe5adb93d1d676 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:50:51.211779 30149 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 279c0ad804164f67aafe5adb93d1d676 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:51.211982 30149 raft_consensus.cc:686] T 00000000000000000000000000000000 P 279c0ad804164f67aafe5adb93d1d676 [term 1 LEADER]: Becoming Leader. State: Replica: 279c0ad804164f67aafe5adb93d1d676, State: Running, Role: LEADER
I20260929 10:50:51.211964 30144 sys_catalog.cc:564] T 00000000000000000000000000000000 P 279c0ad804164f67aafe5adb93d1d676 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:50:51.213168 30149 consensus_queue.cc:237] T 00000000000000000000000000000000 P 279c0ad804164f67aafe5adb93d1d676 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "279c0ad804164f67aafe5adb93d1d676" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44501 } }
I20260929 10:50:51.213833 30150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 279c0ad804164f67aafe5adb93d1d676 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 279c0ad804164f67aafe5adb93d1d676. Latest consensus state: current_term: 1 leader_uuid: "279c0ad804164f67aafe5adb93d1d676" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "279c0ad804164f67aafe5adb93d1d676" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44501 } } }
I20260929 10:50:51.213969 30151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 279c0ad804164f67aafe5adb93d1d676 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "279c0ad804164f67aafe5adb93d1d676" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "279c0ad804164f67aafe5adb93d1d676" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44501 } } }
I20260929 10:50:51.214007 30150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 279c0ad804164f67aafe5adb93d1d676 [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:51.214015 30151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 279c0ad804164f67aafe5adb93d1d676 [sys.catalog]: This master's current role is: LEADER
W20260929 10:50:51.214401 30163 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 279c0ad804164f67aafe5adb93d1d676: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:50:51.214458 30163 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:50:51.214519 30164 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:50:51.214684 30164 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:50:51.216313 30164 catalog_manager.cc:1347] Generated new cluster ID: 22fd4e270e854b8aa2071225b2e13f03
I20260929 10:50:51.216416 30164 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:50:51.238605 30164 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:50:51.239171 30164 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:50:51.246444 30164 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 279c0ad804164f67aafe5adb93d1d676: Generated new TSK 0
I20260929 10:50:51.246667 30164 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:50:51.252929 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.193:0
--local_ip_for_outbound_sockets=127.20.80.193
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:44501
--builtin_ntp_servers=127.20.80.212:44749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260929 10:50:51.354601 30168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:51.354813 30168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:51.354835 30168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:51.355998 30168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:51.356040 30168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:51.356063 30168 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.193
I20260929 10:50:51.357424 30168 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:44749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:44501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.193
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:51.357689 30168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:51.358024 30168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:51.363080 30174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:51.363070 30173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:51.363070 30176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:51.363662 30168 server_base.cc:1034] running on GCE node
I20260929 10:50:51.363879 30168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:51.364140 30168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:51.365306 30168 hybrid_clock.cc:648] HybridClock initialized: now 1790679051365290 us; error 30 us; skew 500 ppm
I20260929 10:50:51.366533 30168 webserver.cc:460] Webserver started at http://127.20.80.193:44757/ using document root <none> and password file <none>
I20260929 10:50:51.366786 30168 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:51.366854 30168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:51.367009 30168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:51.368088 30168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/data/instance:
uuid: "6e6b14a99b3d44499008bf79d7cc53cc"
format_stamp: "Formatted at 2026-09-29 10:50:51 on dist-test-slave-vqch"
server_key: "36919210d4c1127a70e233959b705444"
server_key_iv: "9076a2d074267fcef9dcf6820af19eca"
server_key_version: "encryptionkey@0"
I20260929 10:50:51.368530 30168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance:
uuid: "6e6b14a99b3d44499008bf79d7cc53cc"
format_stamp: "Formatted at 2026-09-29 10:50:51 on dist-test-slave-vqch"
server_key: "36919210d4c1127a70e233959b705444"
server_key_iv: "9076a2d074267fcef9dcf6820af19eca"
server_key_version: "encryptionkey@0"
I20260929 10:50:51.369983 30168 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:50:51.370803 30182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:51.370970 30168 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:51.371071 30168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/wal
uuid: "6e6b14a99b3d44499008bf79d7cc53cc"
format_stamp: "Formatted at 2026-09-29 10:50:51 on dist-test-slave-vqch"
server_key: "36919210d4c1127a70e233959b705444"
server_key_iv: "9076a2d074267fcef9dcf6820af19eca"
server_key_version: "encryptionkey@0"
I20260929 10:50:51.371145 30168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:51.380612 30168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:51.380924 30168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:51.381122 30168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:51.381350 30168 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:51.381675 30168 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:51.381727 30168 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:51.381767 30168 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:51.381795 30168 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:51.387530 30168 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.193:40745
I20260929 10:50:51.387614 30296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.193:40745 every 8 connection(s)
I20260929 10:50:51.387946 30168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
I20260929 10:50:51.388559 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 30168
I20260929 10:50:51.388659 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance
I20260929 10:50:51.388904 20803 external_mini_cluster.cc:1428] Setting key 1cbbb83afeeb38505ac819bfb15a7e6e
I20260929 10:50:51.389997 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.194:0
--local_ip_for_outbound_sockets=127.20.80.194
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:44501
--builtin_ntp_servers=127.20.80.212:44749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260929 10:50:51.394505 30147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:51.388790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:41247 (local address 127.20.80.254:44501)
0929 10:50:51.388863 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:51.388866 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:51.389124 (+ 258us) server_negotiation.cc:407] Connection header received
0929 10:50:51.389812 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:51.389813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:51.389824 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:51.389891 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:51.390646 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:51.391410 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:51.393055 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:51.393239 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:51.393294 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:51.394239 (+ 945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:51.394240 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:51.394240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:50:51.394244 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:51.394257 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:50:51.394261 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:51.394304 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:51.394421 (+ 117us) server_negotiation.cc:299] Negotiation successful
0929 10:50:51.394444 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 10:50:51.394846 30299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:51.388742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44501 (local address 127.20.80.193:41247)
0929 10:50:51.388939 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:51.389010 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:51.389682 (+ 672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:51.389947 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:51.389950 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:51.389958 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:51.390556 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:51.390562 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:51.391516 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:51.391518 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:51.392908 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:51.392911 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:51.393334 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:51.393335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:51.393503 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:51.394073 (+ 570us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:50:51.394074 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:51.394117 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:50:51.394149 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:51.394327 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:51.394334 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:51.394337 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:50:51.394421 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 10:50:51.394546 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
I20260929 10:50:51.395398 30297 heartbeater.cc:347] Connected to a master server at 127.20.80.254:44501
I20260929 10:50:51.395509 30297 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:51.395728 30297 heartbeater.cc:511] Master 127.20.80.254:44501 requested a full tablet report, sending...
I20260929 10:50:51.396271 30107 ts_manager.cc:194] Registered new tserver with Master: 6e6b14a99b3d44499008bf79d7cc53cc (127.20.80.193:40745)
I20260929 10:50:51.396898 30107 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.80.193:41247
W20260929 10:50:51.473363 30300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:51.473624 30300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:51.473650 30300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:51.474928 30300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:51.474967 30300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:51.474994 30300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.194
I20260929 10:50:51.476398 30300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:44749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:44501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.194
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:51.476647 30300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:51.476960 30300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:51.482102 30306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:51.482162 30300 server_base.cc:1034] running on GCE node
W20260929 10:50:51.482097 30308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:51.482125 30305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:51.482496 30300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:51.482733 30300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:51.483906 30300 hybrid_clock.cc:648] HybridClock initialized: now 1790679051483883 us; error 49 us; skew 500 ppm
I20260929 10:50:51.485229 30300 webserver.cc:460] Webserver started at http://127.20.80.194:36461/ using document root <none> and password file <none>
I20260929 10:50:51.485493 30300 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:51.485563 30300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:51.485707 30300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:51.486753 30300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/data/instance:
uuid: "12c0885245174715b96dc044cf235cfd"
format_stamp: "Formatted at 2026-09-29 10:50:51 on dist-test-slave-vqch"
server_key: "cdb7d9a01c0486d6c4482e67886d3c2f"
server_key_iv: "f7771a9c0725b784097282cea8d8115b"
server_key_version: "encryptionkey@0"
I20260929 10:50:51.487154 30300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance:
uuid: "12c0885245174715b96dc044cf235cfd"
format_stamp: "Formatted at 2026-09-29 10:50:51 on dist-test-slave-vqch"
server_key: "cdb7d9a01c0486d6c4482e67886d3c2f"
server_key_iv: "f7771a9c0725b784097282cea8d8115b"
server_key_version: "encryptionkey@0"
I20260929 10:50:51.488482 30300 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:50:51.489249 30314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:51.489409 30300 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:51.489511 30300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/wal
uuid: "12c0885245174715b96dc044cf235cfd"
format_stamp: "Formatted at 2026-09-29 10:50:51 on dist-test-slave-vqch"
server_key: "cdb7d9a01c0486d6c4482e67886d3c2f"
server_key_iv: "f7771a9c0725b784097282cea8d8115b"
server_key_version: "encryptionkey@0"
I20260929 10:50:51.489584 30300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:51.512040 30300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:51.512485 30300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:51.512709 30300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:51.512996 30300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:51.513396 30300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:51.513453 30300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:51.513499 30300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:51.513526 30300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:51.519343 30300 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.194:33743
I20260929 10:50:51.519440 30428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.194:33743 every 8 connection(s)
I20260929 10:50:51.519753 30300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
I20260929 10:50:51.525187 30147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:51.520549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:39689 (local address 127.20.80.254:44501)
0929 10:50:51.520595 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:51.520597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:51.520868 (+ 271us) server_negotiation.cc:407] Connection header received
0929 10:50:51.521496 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:51.521497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:51.521510 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:51.521563 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:51.522245 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:51.522749 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:51.523750 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:51.523949 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:51.524030 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:51.524876 (+ 846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:51.524878 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:51.524878 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:50:51.524883 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:51.524902 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:50:51.524906 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:51.524959 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:51.525080 (+ 121us) server_negotiation.cc:299] Negotiation successful
0929 10:50:51.525108 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 10:50:51.525444 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 30300
I20260929 10:50:51.525537 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance
I20260929 10:50:51.525736 20803 external_mini_cluster.cc:1428] Setting key e79df38a362eacfcee62044da2471605
I20260929 10:50:51.526036 30431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:51.520530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44501 (local address 127.20.80.194:39689)
0929 10:50:51.520744 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:51.520778 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:51.521384 (+ 606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:51.521643 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:51.521647 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:51.521660 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:51.522145 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:51.522149 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:51.522847 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:51.522847 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:51.523635 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:51.523637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:51.524036 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:51.524038 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:51.524176 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:51.524696 (+ 520us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:50:51.524696 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:51.524729 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:50:51.524752 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:51.524965 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:51.524971 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:51.524994 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 10:50:51.525085 (+ 91us) client_negotiation.cc:240] Negotiation successful
0929 10:50:51.525187 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":94,"threads_started":1}
I20260929 10:50:51.526539 30429 heartbeater.cc:347] Connected to a master server at 127.20.80.254:44501
I20260929 10:50:51.526638 30429 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:51.526769 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.195:0
--local_ip_for_outbound_sockets=127.20.80.195
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:44501
--builtin_ntp_servers=127.20.80.212:44749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260929 10:50:51.526835 30429 heartbeater.cc:511] Master 127.20.80.254:44501 requested a full tablet report, sending...
I20260929 10:50:51.527213 30107 ts_manager.cc:194] Registered new tserver with Master: 12c0885245174715b96dc044cf235cfd (127.20.80.194:33743)
I20260929 10:50:51.527592 30107 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.80.194:39689
W20260929 10:50:51.608511 30432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:51.608723 30432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:51.608745 30432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:51.609944 30432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:51.609987 30432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:51.610011 30432 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.195
I20260929 10:50:51.611362 30432 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:44749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:44501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.195
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:51.611727 30432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:51.612082 30432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:51.617411 30440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:51.617393 30437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:51.617471 30438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:51.617583 30432 server_base.cc:1034] running on GCE node
I20260929 10:50:51.617874 30432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:51.618129 30432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:51.619302 30432 hybrid_clock.cc:648] HybridClock initialized: now 1790679051619286 us; error 46 us; skew 500 ppm
I20260929 10:50:51.620484 30432 webserver.cc:460] Webserver started at http://127.20.80.195:37249/ using document root <none> and password file <none>
I20260929 10:50:51.620746 30432 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:51.620821 30432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:51.621035 30432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:51.622119 30432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/data/instance:
uuid: "429204cd52ed471585ff6c566b63f044"
format_stamp: "Formatted at 2026-09-29 10:50:51 on dist-test-slave-vqch"
server_key: "25e1e03e62f2946f4aa97a2e98c7347d"
server_key_iv: "ea29aa697eb2d05dd392783d91326435"
server_key_version: "encryptionkey@0"
I20260929 10:50:51.622481 30432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance:
uuid: "429204cd52ed471585ff6c566b63f044"
format_stamp: "Formatted at 2026-09-29 10:50:51 on dist-test-slave-vqch"
server_key: "25e1e03e62f2946f4aa97a2e98c7347d"
server_key_iv: "ea29aa697eb2d05dd392783d91326435"
server_key_version: "encryptionkey@0"
I20260929 10:50:51.623849 30432 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:50:51.624596 30446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:51.624809 30432 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:51.624935 30432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/wal
uuid: "429204cd52ed471585ff6c566b63f044"
format_stamp: "Formatted at 2026-09-29 10:50:51 on dist-test-slave-vqch"
server_key: "25e1e03e62f2946f4aa97a2e98c7347d"
server_key_iv: "ea29aa697eb2d05dd392783d91326435"
server_key_version: "encryptionkey@0"
I20260929 10:50:51.625051 30432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:51.642211 30432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:51.642500 30432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:51.642604 30432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:51.642782 30432 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:51.643131 30432 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:51.643165 30432 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:51.643222 30432 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:51.643252 30432 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:51.649096 30432 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.195:33375
I20260929 10:50:51.649164 30560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.195:33375 every 8 connection(s)
I20260929 10:50:51.649511 30432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
I20260929 10:50:51.651813 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 30432
I20260929 10:50:51.651939 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance
I20260929 10:50:51.652153 20803 external_mini_cluster.cc:1428] Setting key 0fcbca1448d8be4560835004b2ed1e57
I20260929 10:50:51.655177 30147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:51.650248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:40145 (local address 127.20.80.254:44501)
0929 10:50:51.650328 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:51.650331 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:51.650556 (+ 225us) server_negotiation.cc:407] Connection header received
0929 10:50:51.651052 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:51.651053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:51.651064 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:51.651130 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:51.651890 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:51.652651 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:51.653831 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:51.654017 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:51.654074 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:51.654867 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:51.654868 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:51.654868 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:50:51.654872 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:51.654888 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:50:51.654891 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:51.654937 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:51.655088 (+ 151us) server_negotiation.cc:299] Negotiation successful
0929 10:50:51.655119 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 10:50:51.655462 30563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:51.650221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44501 (local address 127.20.80.195:40145)
0929 10:50:51.650422 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:51.650464 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:51.650955 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:51.651212 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:51.651216 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:51.651224 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:51.651745 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:51.651750 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:51.652759 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:51.652760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:51.653729 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:51.653732 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:51.654094 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:51.654095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:51.654203 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:51.654719 (+ 516us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:50:51.654720 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:51.654748 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:50:51.654770 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:51.654995 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:51.655002 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:51.655004 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 10:50:51.655084 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 10:50:51.655197 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":85,"threads_started":1}
I20260929 10:50:51.655983 30561 heartbeater.cc:347] Connected to a master server at 127.20.80.254:44501
I20260929 10:50:51.656080 30561 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:51.656224 30561 heartbeater.cc:511] Master 127.20.80.254:44501 requested a full tablet report, sending...
I20260929 10:50:51.656563 30107 ts_manager.cc:194] Registered new tserver with Master: 429204cd52ed471585ff6c566b63f044 (127.20.80.195:33375)
I20260929 10:50:51.656912 30107 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.80.195:40145
I20260929 10:50:51.664106 20803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 10:50:51.666060 30147 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:51.665498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38488 (local address 127.20.80.254:44501)
0929 10:50:51.665560 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:51.665562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:51.665694 (+ 132us) server_negotiation.cc:407] Connection header received
0929 10:50:51.665749 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:51.665750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:51.665759 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:51.665846 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:51.665968 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38488: BlockingRecv error: recv got EOF from 127.0.0.1:38488 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 10:50:51.666152 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 30168
I20260929 10:50:51.671063 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 30300
I20260929 10:50:51.675684 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 30432
I20260929 10:50:51.680080 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 30076
2026-09-29T10:50:51Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (626 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-29T10:50:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:50:51Z Disabled control of system clock
I20260929 10:50:51.707193 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:37795
--webserver_interface=127.20.80.254
--webserver_port=0
--builtin_ntp_servers=127.20.80.212:42181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:37795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 10:50:51.787642 30572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:51.787911 30572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:51.787936 30572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:51.789216 30572 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:50:51.789269 30572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:51.789285 30572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:51.789299 30572 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:50:51.789314 30572 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:50:51.790678 30572 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:42181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:37795
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:37795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:51.790899 30572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:51.791222 30572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:51.796289 30580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:51.796314 30578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:51.796309 30577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:51.796460 30572 server_base.cc:1034] running on GCE node
I20260929 10:50:51.796707 30572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:51.797055 30572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:51.798233 30572 hybrid_clock.cc:648] HybridClock initialized: now 1790679051798224 us; error 38 us; skew 500 ppm
I20260929 10:50:51.799574 30572 webserver.cc:460] Webserver started at http://127.20.80.254:41825/ using document root <none> and password file <none>
I20260929 10:50:51.799858 30572 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:51.799933 30572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:51.800099 30572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:51.801313 30572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "dae478da87ac4ca0a94a610310fd31ca"
format_stamp: "Formatted at 2026-09-29 10:50:51 on dist-test-slave-vqch"
server_key: "7af1ad87a4e8dbcfcb3bdf665b4bd591"
server_key_iv: "8fde872fb3e10f912338bc4a338fb709"
server_key_version: "encryptionkey@0"
I20260929 10:50:51.801684 30572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "dae478da87ac4ca0a94a610310fd31ca"
format_stamp: "Formatted at 2026-09-29 10:50:51 on dist-test-slave-vqch"
server_key: "7af1ad87a4e8dbcfcb3bdf665b4bd591"
server_key_iv: "8fde872fb3e10f912338bc4a338fb709"
server_key_version: "encryptionkey@0"
I20260929 10:50:51.803000 30572 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:51.803783 30586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:51.803989 30572 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:51.804095 30572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "dae478da87ac4ca0a94a610310fd31ca"
format_stamp: "Formatted at 2026-09-29 10:50:51 on dist-test-slave-vqch"
server_key: "7af1ad87a4e8dbcfcb3bdf665b4bd591"
server_key_iv: "8fde872fb3e10f912338bc4a338fb709"
server_key_version: "encryptionkey@0"
I20260929 10:50:51.804188 30572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:51.807658 30572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:51.807952 30572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:51.808104 30572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:51.812457 30572 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:37795
I20260929 10:50:51.812507 30639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:37795 every 8 connection(s)
I20260929 10:50:51.812865 30572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:50:51.813649 30640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:51.816561 30640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dae478da87ac4ca0a94a610310fd31ca: Bootstrap starting.
I20260929 10:50:51.817237 30640 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dae478da87ac4ca0a94a610310fd31ca: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:51.817538 30640 log.cc:824] T 00000000000000000000000000000000 P dae478da87ac4ca0a94a610310fd31ca: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:51.818272 30640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dae478da87ac4ca0a94a610310fd31ca: No bootstrap required, opened a new log
I20260929 10:50:51.819892 30640 raft_consensus.cc:348] T 00000000000000000000000000000000 P dae478da87ac4ca0a94a610310fd31ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dae478da87ac4ca0a94a610310fd31ca" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37795 } }
I20260929 10:50:51.820037 30640 raft_consensus.cc:374] T 00000000000000000000000000000000 P dae478da87ac4ca0a94a610310fd31ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:51.820060 30640 raft_consensus.cc:729] T 00000000000000000000000000000000 P dae478da87ac4ca0a94a610310fd31ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dae478da87ac4ca0a94a610310fd31ca, State: Initialized, Role: FOLLOWER
I20260929 10:50:51.820237 30640 consensus_queue.cc:260] T 00000000000000000000000000000000 P dae478da87ac4ca0a94a610310fd31ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dae478da87ac4ca0a94a610310fd31ca" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37795 } }
I20260929 10:50:51.820302 30640 raft_consensus.cc:388] T 00000000000000000000000000000000 P dae478da87ac4ca0a94a610310fd31ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:50:51.820324 30640 raft_consensus.cc:482] T 00000000000000000000000000000000 P dae478da87ac4ca0a94a610310fd31ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:50:51.820390 30640 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dae478da87ac4ca0a94a610310fd31ca [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:51.820927 30640 raft_consensus.cc:504] T 00000000000000000000000000000000 P dae478da87ac4ca0a94a610310fd31ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dae478da87ac4ca0a94a610310fd31ca" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37795 } }
I20260929 10:50:51.821075 30640 leader_election.cc:302] T 00000000000000000000000000000000 P dae478da87ac4ca0a94a610310fd31ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dae478da87ac4ca0a94a610310fd31ca; no voters:
I20260929 10:50:51.821278 30640 leader_election.cc:288] T 00000000000000000000000000000000 P dae478da87ac4ca0a94a610310fd31ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:50:51.821349 30643 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dae478da87ac4ca0a94a610310fd31ca [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:51.821539 30643 raft_consensus.cc:686] T 00000000000000000000000000000000 P dae478da87ac4ca0a94a610310fd31ca [term 1 LEADER]: Becoming Leader. State: Replica: dae478da87ac4ca0a94a610310fd31ca, State: Running, Role: LEADER
I20260929 10:50:51.821581 30640 sys_catalog.cc:564] T 00000000000000000000000000000000 P dae478da87ac4ca0a94a610310fd31ca [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:50:51.821583 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 30572
I20260929 10:50:51.821664 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:50:51.821640 30643 consensus_queue.cc:237] T 00000000000000000000000000000000 P dae478da87ac4ca0a94a610310fd31ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dae478da87ac4ca0a94a610310fd31ca" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37795 } }
I20260929 10:50:51.821892 20803 external_mini_cluster.cc:1428] Setting key 50db87ad8ec2f1e5e111f54c7161ffbb
I20260929 10:50:51.821980 30643 sys_catalog.cc:455] T 00000000000000000000000000000000 P dae478da87ac4ca0a94a610310fd31ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader dae478da87ac4ca0a94a610310fd31ca. Latest consensus state: current_term: 1 leader_uuid: "dae478da87ac4ca0a94a610310fd31ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dae478da87ac4ca0a94a610310fd31ca" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37795 } } }
I20260929 10:50:51.821988 30645 sys_catalog.cc:455] T 00000000000000000000000000000000 P dae478da87ac4ca0a94a610310fd31ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dae478da87ac4ca0a94a610310fd31ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dae478da87ac4ca0a94a610310fd31ca" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37795 } } }
I20260929 10:50:51.822065 30643 sys_catalog.cc:458] T 00000000000000000000000000000000 P dae478da87ac4ca0a94a610310fd31ca [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:51.822088 30645 sys_catalog.cc:458] T 00000000000000000000000000000000 P dae478da87ac4ca0a94a610310fd31ca [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:51.823136 30654 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:50:51.823642 30654 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:50:51.824474 30652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:51.822473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57874 (local address 127.20.80.254:37795)
0929 10:50:51.822771 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:51.822779 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:51.822814 (+ 35us) server_negotiation.cc:407] Connection header received
0929 10:50:51.822934 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:51.822941 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:51.822995 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:51.823687 (+ 692us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:51.824074 (+ 387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:51.824079 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:51.824079 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:50:51.824083 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:51.824119 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:50:51.824133 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:51.824208 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:51.824251 (+ 43us) server_negotiation.cc:299] Negotiation successful
0929 10:50:51.824306 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":97,"threads_started":1}
I20260929 10:50:51.825186 30654 catalog_manager.cc:1347] Generated new cluster ID: ee933f4935eb460bae074e5bce13395a
I20260929 10:50:51.825240 30654 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:50:51.845372 30654 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:50:51.845605 30654 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:50:51.850798 30654 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dae478da87ac4ca0a94a610310fd31ca: Generated new TSK 0
I20260929 10:50:51.851030 30654 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:50:51.856236 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.193:0
--local_ip_for_outbound_sockets=127.20.80.193
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37795
--builtin_ntp_servers=127.20.80.212:42181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 10:50:51.959375 30664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:51.959574 30664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:51.959597 30664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:51.960764 30664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:51.960803 30664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:51.960829 30664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.193
I20260929 10:50:51.962203 30664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:42181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.193
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:51.962443 30664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:51.962742 30664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:51.967795 30670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:51.967839 30672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:51.967998 30664 server_base.cc:1034] running on GCE node
W20260929 10:50:51.967773 30669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:51.968380 30664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:51.968662 30664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:51.969857 30664 hybrid_clock.cc:648] HybridClock initialized: now 1790679051969836 us; error 37 us; skew 500 ppm
I20260929 10:50:51.971257 30664 webserver.cc:460] Webserver started at http://127.20.80.193:40837/ using document root <none> and password file <none>
I20260929 10:50:51.971508 30664 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:51.971571 30664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:51.971695 30664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:51.973091 30664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/data/instance:
uuid: "a43b1c6e2f4f4447a91b0e5e63c4752a"
format_stamp: "Formatted at 2026-09-29 10:50:51 on dist-test-slave-vqch"
server_key: "e0fc996d9c88ba003db1b941332c9926"
server_key_iv: "2dcc09045e47792fab37d3c47ea6c8c1"
server_key_version: "encryptionkey@0"
I20260929 10:50:51.973476 30664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance:
uuid: "a43b1c6e2f4f4447a91b0e5e63c4752a"
format_stamp: "Formatted at 2026-09-29 10:50:51 on dist-test-slave-vqch"
server_key: "e0fc996d9c88ba003db1b941332c9926"
server_key_iv: "2dcc09045e47792fab37d3c47ea6c8c1"
server_key_version: "encryptionkey@0"
I20260929 10:50:51.975135 30664 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:50:51.975996 30678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:51.976191 30664 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:51.976295 30664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/wal
uuid: "a43b1c6e2f4f4447a91b0e5e63c4752a"
format_stamp: "Formatted at 2026-09-29 10:50:51 on dist-test-slave-vqch"
server_key: "e0fc996d9c88ba003db1b941332c9926"
server_key_iv: "2dcc09045e47792fab37d3c47ea6c8c1"
server_key_version: "encryptionkey@0"
I20260929 10:50:51.976370 30664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:51.979689 30664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:51.979970 30664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:51.980111 30664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:51.980320 30664 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:51.980618 30664 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:51.980671 30664 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:51.980720 30664 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:51.980751 30664 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:51.986135 30664 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.193:41215
I20260929 10:50:51.986222 30791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.193:41215 every 8 connection(s)
I20260929 10:50:51.986549 30664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
I20260929 10:50:51.989639 30652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:51.987310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:54599 (local address 127.20.80.254:37795)
0929 10:50:51.987370 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:51.987373 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:51.987614 (+ 241us) server_negotiation.cc:407] Connection header received
0929 10:50:51.988058 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:51.988060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:51.988067 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:51.988146 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:51.989396 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:51.989397 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:51.989397 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:50:51.989402 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:51.989419 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:50:51.989424 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:51.989492 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:51.989560 (+ 68us) server_negotiation.cc:299] Negotiation successful
0929 10:50:51.989574 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260929 10:50:51.989786 30794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:51.987304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37795 (local address 127.20.80.193:54599)
0929 10:50:51.987477 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:51.987516 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:51.987966 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:51.988217 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:51.988220 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:51.988228 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:51.989254 (+ 1026us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:50:51.989254 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:51.989282 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:50:51.989311 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:51.989490 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:51.989492 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:51.989495 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:50:51.989545 (+ 50us) client_negotiation.cc:240] Negotiation successful
0929 10:50:51.989590 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
I20260929 10:50:51.990268 30792 heartbeater.cc:347] Connected to a master server at 127.20.80.254:37795
I20260929 10:50:51.990362 30792 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:51.990504 30792 heartbeater.cc:511] Master 127.20.80.254:37795 requested a full tablet report, sending...
I20260929 10:50:51.991016 30603 ts_manager.cc:194] Registered new tserver with Master: a43b1c6e2f4f4447a91b0e5e63c4752a (127.20.80.193:41215)
I20260929 10:50:51.992223 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 30664
I20260929 10:50:51.992297 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance
I20260929 10:50:51.992513 20803 external_mini_cluster.cc:1428] Setting key cad6b347b6a2902a179b936b1906b30c
I20260929 10:50:51.993541 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.194:0
--local_ip_for_outbound_sockets=127.20.80.194
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37795
--builtin_ntp_servers=127.20.80.212:42181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 10:50:52.074721 30795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:52.074932 30795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:52.074954 30795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:52.076119 30795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:52.076159 30795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:52.076216 30795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.194
I20260929 10:50:52.077654 30795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:42181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.194
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:52.077884 30795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:52.078189 30795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:52.083460 30801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:52.083550 30803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:52.083456 30800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:52.083616 30795 server_base.cc:1034] running on GCE node
I20260929 10:50:52.083878 30795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:52.084095 30795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:52.085289 30795 hybrid_clock.cc:648] HybridClock initialized: now 1790679052085276 us; error 49 us; skew 500 ppm
I20260929 10:50:52.086568 30795 webserver.cc:460] Webserver started at http://127.20.80.194:38205/ using document root <none> and password file <none>
I20260929 10:50:52.086822 30795 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:52.086894 30795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:52.087040 30795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:52.088196 30795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/data/instance:
uuid: "46aa3b76df8c4b81bb266eaf02898ccb"
format_stamp: "Formatted at 2026-09-29 10:50:52 on dist-test-slave-vqch"
server_key: "d880642c15be254941cf8bcdb9245402"
server_key_iv: "d9e24928b83217f468b86b643de565c9"
server_key_version: "encryptionkey@0"
I20260929 10:50:52.088572 30795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance:
uuid: "46aa3b76df8c4b81bb266eaf02898ccb"
format_stamp: "Formatted at 2026-09-29 10:50:52 on dist-test-slave-vqch"
server_key: "d880642c15be254941cf8bcdb9245402"
server_key_iv: "d9e24928b83217f468b86b643de565c9"
server_key_version: "encryptionkey@0"
I20260929 10:50:52.090006 30795 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 10:50:52.090752 30809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:52.091007 30795 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:50:52.091135 30795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/wal
uuid: "46aa3b76df8c4b81bb266eaf02898ccb"
format_stamp: "Formatted at 2026-09-29 10:50:52 on dist-test-slave-vqch"
server_key: "d880642c15be254941cf8bcdb9245402"
server_key_iv: "d9e24928b83217f468b86b643de565c9"
server_key_version: "encryptionkey@0"
I20260929 10:50:52.091221 30795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:52.094712 30795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:52.094966 30795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:52.095103 30795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:52.095305 30795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:52.095602 30795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:52.095652 30795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:52.095698 30795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:52.095726 30795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:52.101558 30795 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.194:40753
I20260929 10:50:52.101606 30924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.194:40753 every 8 connection(s)
I20260929 10:50:52.101923 30795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
I20260929 10:50:52.104693 30652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:52.102627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:42911 (local address 127.20.80.254:37795)
0929 10:50:52.102693 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:52.102695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:52.102905 (+ 210us) server_negotiation.cc:407] Connection header received
0929 10:50:52.103356 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:52.103358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:52.103363 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:52.103429 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:52.104491 (+ 1062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:52.104493 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:52.104493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:50:52.104497 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:52.104512 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:50:52.104516 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:52.104569 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:52.104629 (+ 60us) server_negotiation.cc:299] Negotiation successful
0929 10:50:52.104643 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 10:50:52.104924 30927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:52.102608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37795 (local address 127.20.80.194:42911)
0929 10:50:52.102783 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:52.102823 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:52.103270 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:52.103483 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:52.103487 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:52.103495 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:52.104352 (+ 857us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:50:52.104353 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:52.104379 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:50:52.104405 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:52.104570 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:52.104572 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:52.104574 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 10:50:52.104624 (+ 50us) client_negotiation.cc:240] Negotiation successful
0929 10:50:52.104716 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":80,"threads_started":1}
I20260929 10:50:52.105456 30925 heartbeater.cc:347] Connected to a master server at 127.20.80.254:37795
I20260929 10:50:52.105531 30925 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:52.105710 30925 heartbeater.cc:511] Master 127.20.80.254:37795 requested a full tablet report, sending...
I20260929 10:50:52.106079 30603 ts_manager.cc:194] Registered new tserver with Master: 46aa3b76df8c4b81bb266eaf02898ccb (127.20.80.194:40753)
I20260929 10:50:52.108019 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 30795
I20260929 10:50:52.108121 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance
I20260929 10:50:52.108312 20803 external_mini_cluster.cc:1428] Setting key f2aa4e063f940f636be5a1e7930e7e28
I20260929 10:50:52.109320 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.195:0
--local_ip_for_outbound_sockets=127.20.80.195
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37795
--builtin_ntp_servers=127.20.80.212:42181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 10:50:52.196950 30928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:52.197228 30928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:52.197256 30928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:52.198398 30928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:52.198437 30928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:52.198464 30928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.195
I20260929 10:50:52.199807 30928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:42181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.195
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:52.200008 30928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:52.200359 30928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:52.205449 30936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:52.205448 30934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:52.205523 30933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:52.205600 30928 server_base.cc:1034] running on GCE node
I20260929 10:50:52.205911 30928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:52.206168 30928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:52.207345 30928 hybrid_clock.cc:648] HybridClock initialized: now 1790679052207331 us; error 27 us; skew 500 ppm
I20260929 10:50:52.208665 30928 webserver.cc:460] Webserver started at http://127.20.80.195:39083/ using document root <none> and password file <none>
I20260929 10:50:52.209054 30928 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:52.209142 30928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:52.209295 30928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:52.210426 30928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/data/instance:
uuid: "f79beb493eeb4a57bddfd2557c582ab8"
format_stamp: "Formatted at 2026-09-29 10:50:52 on dist-test-slave-vqch"
server_key: "1dcc1854c97c286b1e0d6059d63d9f85"
server_key_iv: "f5bc82e12aaeed1a9972f5903a12e882"
server_key_version: "encryptionkey@0"
I20260929 10:50:52.210819 30928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance:
uuid: "f79beb493eeb4a57bddfd2557c582ab8"
format_stamp: "Formatted at 2026-09-29 10:50:52 on dist-test-slave-vqch"
server_key: "1dcc1854c97c286b1e0d6059d63d9f85"
server_key_iv: "f5bc82e12aaeed1a9972f5903a12e882"
server_key_version: "encryptionkey@0"
I20260929 10:50:52.212250 30928 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 10:50:52.213076 30942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:52.213292 30928 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:50:52.213402 30928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/wal
uuid: "f79beb493eeb4a57bddfd2557c582ab8"
format_stamp: "Formatted at 2026-09-29 10:50:52 on dist-test-slave-vqch"
server_key: "1dcc1854c97c286b1e0d6059d63d9f85"
server_key_iv: "f5bc82e12aaeed1a9972f5903a12e882"
server_key_version: "encryptionkey@0"
I20260929 10:50:52.213495 30928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:52.217147 30928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:52.217429 30928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:52.217586 30928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:52.217809 30928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:52.218127 30928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:52.218186 30928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:52.218223 30928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:52.218252 30928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:52.224854 30928 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.195:37227
I20260929 10:50:52.224888 31057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.195:37227 every 8 connection(s)
I20260929 10:50:52.225289 30928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
I20260929 10:50:52.228163 30652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:52.226041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:37305 (local address 127.20.80.254:37795)
0929 10:50:52.226085 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:52.226087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:52.226297 (+ 210us) server_negotiation.cc:407] Connection header received
0929 10:50:52.226824 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:52.226825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:52.226831 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:52.226881 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:52.227974 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:52.227976 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:52.227976 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:50:52.227980 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:52.227994 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:50:52.227998 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:52.228045 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:52.228098 (+ 53us) server_negotiation.cc:299] Negotiation successful
0929 10:50:52.228110 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 10:50:52.228402 31060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:52.226027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37795 (local address 127.20.80.195:37305)
0929 10:50:52.226196 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:52.226231 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:52.226678 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:52.226938 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:52.226941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:52.226949 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:52.227835 (+ 886us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:50:52.227835 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:52.227863 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:50:52.227891 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:52.228043 (+ 152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:52.228046 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:52.228049 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:50:52.228098 (+ 49us) client_negotiation.cc:240] Negotiation successful
0929 10:50:52.228146 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":84,"threads_started":1}
I20260929 10:50:52.229009 31058 heartbeater.cc:347] Connected to a master server at 127.20.80.254:37795
I20260929 10:50:52.229103 31058 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:52.229221 31058 heartbeater.cc:511] Master 127.20.80.254:37795 requested a full tablet report, sending...
I20260929 10:50:52.229583 30603 ts_manager.cc:194] Registered new tserver with Master: f79beb493eeb4a57bddfd2557c582ab8 (127.20.80.195:37227)
I20260929 10:50:52.234150 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 30928
I20260929 10:50:52.234229 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance
I20260929 10:50:52.234386 20803 external_mini_cluster.cc:1428] Setting key 37e6327ee356024134274a73fc17b5af
I20260929 10:50:52.235383 20803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 10:50:52.237319 30652 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:50:52.236759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57886 (local address 127.20.80.254:37795)
0929 10:50:52.236841 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:52.236842 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:52.236937 (+ 95us) server_negotiation.cc:407] Connection header received
0929 10:50:52.237059 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:52.237060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:52.237066 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:52.237122 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:52.237242 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57886: BlockingRecv error: recv got EOF from 127.0.0.1:57886 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 10:50:52.237363 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 30664
I20260929 10:50:52.242277 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 30795
I20260929 10:50:52.246639 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 30928
I20260929 10:50:52.251144 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 30572
2026-09-29T10:50:52Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (570 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:50:52 dist-test-slave-vqch krb5kdc[31068](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:50:52 dist-test-slave-vqch krb5kdc[31068](info): set up 2 sockets
Sep 29 10:50:52 dist-test-slave-vqch krb5kdc[31068](info): commencing operation
krb5kdc: starting...
W20260929 10:50:53.289609 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:50:53 dist-test-slave-vqch krb5kdc[31068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:50:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:50:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.80.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
I20260929 10:50:53.412086 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:46581
--webserver_interface=127.20.80.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:43863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:46581
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:53.493070 31081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:53.493309 31081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:53.493343 31081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:53.494503 31081 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:50:53.494575 31081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:53.494598 31081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:53.494627 31081 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:50:53.494657 31081 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:50:53.496146 31081 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:43863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:46581
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:46581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:53.496420 31081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:53.496647 31081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:53.501636 31086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:53.501725 31087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:53.501823 31081 server_base.cc:1034] running on GCE node
W20260929 10:50:53.501634 31089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:53.502198 31081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:53.502575 31081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:53.503822 31081 hybrid_clock.cc:648] HybridClock initialized: now 1790679053503782 us; error 51 us; skew 500 ppm
Sep 29 10:50:53 dist-test-slave-vqch krb5kdc[31068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679053, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:53.506657 31081 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:50:53.507079 31081 webserver.cc:460] Webserver started at http://127.20.80.254:41887/ using document root <none> and password file <none>
I20260929 10:50:53.507311 31081 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:53.507357 31081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:53.507519 31081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:53.508592 31081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "c50d94a7bcb9489d80a93e7f466a93fc"
format_stamp: "Formatted at 2026-09-29 10:50:53 on dist-test-slave-vqch"
server_key: "cfaf27bbccd6c3b7afed762e0b49bba1"
server_key_iv: "b88e048ce36da8a6cde97ae112919246"
server_key_version: "encryptionkey@0"
I20260929 10:50:53.508945 31081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "c50d94a7bcb9489d80a93e7f466a93fc"
format_stamp: "Formatted at 2026-09-29 10:50:53 on dist-test-slave-vqch"
server_key: "cfaf27bbccd6c3b7afed762e0b49bba1"
server_key_iv: "b88e048ce36da8a6cde97ae112919246"
server_key_version: "encryptionkey@0"
I20260929 10:50:53.510308 31081 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:50:53.511117 31096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:53.511317 31081 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:50:53.511425 31081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "c50d94a7bcb9489d80a93e7f466a93fc"
format_stamp: "Formatted at 2026-09-29 10:50:53 on dist-test-slave-vqch"
server_key: "cfaf27bbccd6c3b7afed762e0b49bba1"
server_key_iv: "b88e048ce36da8a6cde97ae112919246"
server_key_version: "encryptionkey@0"
I20260929 10:50:53.511498 31081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:53.522544 31081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:53.524464 31081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:53.524612 31081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:53.528703 31081 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:46581
I20260929 10:50:53.528764 31148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:46581 every 8 connection(s)
I20260929 10:50:53.529104 31081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:50:53.530092 31149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:53.532335 31149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c50d94a7bcb9489d80a93e7f466a93fc: Bootstrap starting.
I20260929 10:50:53.532889 31149 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c50d94a7bcb9489d80a93e7f466a93fc: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:53.533201 31149 log.cc:824] T 00000000000000000000000000000000 P c50d94a7bcb9489d80a93e7f466a93fc: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:53.533879 31149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c50d94a7bcb9489d80a93e7f466a93fc: No bootstrap required, opened a new log
I20260929 10:50:53.535470 31149 raft_consensus.cc:348] T 00000000000000000000000000000000 P c50d94a7bcb9489d80a93e7f466a93fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c50d94a7bcb9489d80a93e7f466a93fc" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46581 } }
I20260929 10:50:53.535614 31149 raft_consensus.cc:374] T 00000000000000000000000000000000 P c50d94a7bcb9489d80a93e7f466a93fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:53.535636 31149 raft_consensus.cc:729] T 00000000000000000000000000000000 P c50d94a7bcb9489d80a93e7f466a93fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c50d94a7bcb9489d80a93e7f466a93fc, State: Initialized, Role: FOLLOWER
I20260929 10:50:53.535851 31149 consensus_queue.cc:260] T 00000000000000000000000000000000 P c50d94a7bcb9489d80a93e7f466a93fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c50d94a7bcb9489d80a93e7f466a93fc" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46581 } }
I20260929 10:50:53.535943 31149 raft_consensus.cc:388] T 00000000000000000000000000000000 P c50d94a7bcb9489d80a93e7f466a93fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:50:53.535965 31149 raft_consensus.cc:482] T 00000000000000000000000000000000 P c50d94a7bcb9489d80a93e7f466a93fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:50:53.535996 31149 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c50d94a7bcb9489d80a93e7f466a93fc [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:53.536805 31149 raft_consensus.cc:504] T 00000000000000000000000000000000 P c50d94a7bcb9489d80a93e7f466a93fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c50d94a7bcb9489d80a93e7f466a93fc" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46581 } }
I20260929 10:50:53.536899 31149 leader_election.cc:302] T 00000000000000000000000000000000 P c50d94a7bcb9489d80a93e7f466a93fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c50d94a7bcb9489d80a93e7f466a93fc; no voters:
I20260929 10:50:53.537102 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 31081
I20260929 10:50:53.537130 31149 leader_election.cc:288] T 00000000000000000000000000000000 P c50d94a7bcb9489d80a93e7f466a93fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:50:53.537161 31152 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c50d94a7bcb9489d80a93e7f466a93fc [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:53.537220 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:50:53.537415 31149 sys_catalog.cc:564] T 00000000000000000000000000000000 P c50d94a7bcb9489d80a93e7f466a93fc [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:50:53.537472 20803 external_mini_cluster.cc:1428] Setting key e5850d91e6fce99d85c75c042163918b
I20260929 10:50:53.537412 31152 raft_consensus.cc:686] T 00000000000000000000000000000000 P c50d94a7bcb9489d80a93e7f466a93fc [term 1 LEADER]: Becoming Leader. State: Replica: c50d94a7bcb9489d80a93e7f466a93fc, State: Running, Role: LEADER
I20260929 10:50:53.538903 31152 consensus_queue.cc:237] T 00000000000000000000000000000000 P c50d94a7bcb9489d80a93e7f466a93fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c50d94a7bcb9489d80a93e7f466a93fc" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46581 } }
I20260929 10:50:53.539670 31154 sys_catalog.cc:455] T 00000000000000000000000000000000 P c50d94a7bcb9489d80a93e7f466a93fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c50d94a7bcb9489d80a93e7f466a93fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c50d94a7bcb9489d80a93e7f466a93fc" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46581 } } }
I20260929 10:50:53.539740 31154 sys_catalog.cc:458] T 00000000000000000000000000000000 P c50d94a7bcb9489d80a93e7f466a93fc [sys.catalog]: This master's current role is: LEADER
I20260929 10:50:53.539522 31153 sys_catalog.cc:455] T 00000000000000000000000000000000 P c50d94a7bcb9489d80a93e7f466a93fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader c50d94a7bcb9489d80a93e7f466a93fc. Latest consensus state: current_term: 1 leader_uuid: "c50d94a7bcb9489d80a93e7f466a93fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c50d94a7bcb9489d80a93e7f466a93fc" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46581 } } }
W20260929 10:50:53.539968 31169 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c50d94a7bcb9489d80a93e7f466a93fc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260929 10:50:53.539986 31153 sys_catalog.cc:458] T 00000000000000000000000000000000 P c50d94a7bcb9489d80a93e7f466a93fc [sys.catalog]: This master's current role is: LEADER
W20260929 10:50:53.540078 31169 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:50:53.540148 31171 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:50:53.540339 31171 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:50:53.542057 31171 catalog_manager.cc:1347] Generated new cluster ID: 312deaf3441f4b0b9a3b939f482aac9d
I20260929 10:50:53.542111 31171 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 10:50:53 dist-test-slave-vqch krb5kdc[31068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:53.550084 31158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:53.538023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37124 (local address 127.20.80.254:46581)
0929 10:50:53.538408 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:53.538417 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:53.538453 (+ 36us) server_negotiation.cc:407] Connection header received
0929 10:50:53.538592 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:53.538600 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:53.538668 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:53.539685 (+ 1017us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:53.539804 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:53.541207 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:53.542247 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:53.542836 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:53.542928 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:53.545757 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:53.545762 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:53.545763 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:53.545777 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:53.548385 (+ 2608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:53.548771 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:53.548774 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:53.548774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:53.548896 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:53.549136 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:53.549137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:53.549137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:53.549479 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:53.549648 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:53.549719 (+ 71us) server_negotiation.cc:299] Negotiation successful
0929 10:50:53.549858 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":93,"threads_started":1}
I20260929 10:50:53.563264 31171 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:50:53.563988 31171 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:50:53.567930 31171 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c50d94a7bcb9489d80a93e7f466a93fc: Generated new TSK 0
I20260929 10:50:53.568104 31171 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.80.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
I20260929 10:50:53.595587 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.193:0
--local_ip_for_outbound_sockets=127.20.80.193
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:46581
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:43863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:53.679366 31176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:53.679606 31176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:53.679643 31176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:53.680816 31176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:53.680884 31176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:53.680929 31176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.193
I20260929 10:50:53.682480 31176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:43863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:46581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.193
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:53.682762 31176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:53.682998 31176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 10:50:53.688153 31176 server_base.cc:1034] running on GCE node
W20260929 10:50:53.688135 31184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:53.688134 31182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:53.688119 31181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:53.688603 31176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:53.688889 31176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:53.690086 31176 hybrid_clock.cc:648] HybridClock initialized: now 1790679053690072 us; error 35 us; skew 500 ppm
Sep 29 10:50:53 dist-test-slave-vqch krb5kdc[31068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679053, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:53.692946 31176 init.cc:377] Logged in from keytab as kudu/127.20.80.193@KRBTEST.COM (short username kudu)
I20260929 10:50:53.693465 31176 webserver.cc:460] Webserver started at http://127.20.80.193:42177/ using document root <none> and password file <none>
I20260929 10:50:53.693676 31176 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:53.693720 31176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:53.693814 31176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:53.694826 31176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/data/instance:
uuid: "eb0dfb4ba76246b685d61ffd228456e3"
format_stamp: "Formatted at 2026-09-29 10:50:53 on dist-test-slave-vqch"
server_key: "e812606c468c5cf023f87cce9162e11c"
server_key_iv: "680efa485187dabcd0b73ccd6ce34948"
server_key_version: "encryptionkey@0"
I20260929 10:50:53.695147 31176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance:
uuid: "eb0dfb4ba76246b685d61ffd228456e3"
format_stamp: "Formatted at 2026-09-29 10:50:53 on dist-test-slave-vqch"
server_key: "e812606c468c5cf023f87cce9162e11c"
server_key_iv: "680efa485187dabcd0b73ccd6ce34948"
server_key_version: "encryptionkey@0"
I20260929 10:50:53.696478 31176 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:50:53.697305 31191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:53.697517 31176 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:50:53.697599 31176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/wal
uuid: "eb0dfb4ba76246b685d61ffd228456e3"
format_stamp: "Formatted at 2026-09-29 10:50:53 on dist-test-slave-vqch"
server_key: "e812606c468c5cf023f87cce9162e11c"
server_key_iv: "680efa485187dabcd0b73ccd6ce34948"
server_key_version: "encryptionkey@0"
I20260929 10:50:53.697654 31176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:53.714267 31176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:53.716380 31176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:53.716491 31176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:53.716668 31176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:53.717061 31176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:53.717099 31176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:53.717123 31176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:53.717165 31176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:53.722642 31176 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.193:34685
I20260929 10:50:53.722702 31304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.193:34685 every 8 connection(s)
I20260929 10:50:53.723016 31176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
Sep 29 10:50:53 dist-test-slave-vqch krb5kdc[31068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679053, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:53.731019 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 31176
I20260929 10:50:53.731096 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance
I20260929 10:50:53.731333 20803 external_mini_cluster.cc:1428] Setting key c2384a466ca676da09d256e4bb48cb36
I20260929 10:50:53.732474 31158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:53.723725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:32783 (local address 127.20.80.254:46581)
0929 10:50:53.723847 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:53.723850 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:53.723960 (+ 110us) server_negotiation.cc:407] Connection header received
0929 10:50:53.724442 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:53.724443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:53.724452 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:53.724510 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:53.725417 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:53.725913 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:53.726881 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:53.727081 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:53.727132 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:53.729308 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:53.729309 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:53.729309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:53.729318 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:53.730767 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:53.731276 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:53.731277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:53.731278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:53.731343 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:53.731690 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:53.731691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:53.731691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:53.731832 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:53.731903 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:53.732239 (+ 336us) server_negotiation.cc:299] Negotiation successful
0929 10:50:53.732339 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260929 10:50:53.732620 31307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:53.723678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:46581 (local address 127.20.80.193:32783)
0929 10:50:53.723822 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:53.723872 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:53.724357 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:53.724565 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:53.724568 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:53.725145 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:53.725340 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:53.725344 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:53.725996 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:53.725997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:53.726801 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:53.726803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:53.727153 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:53.727154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:53.727262 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:53.727708 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:53.727716 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:53.729216 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:53.730860 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:53.730867 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:53.730869 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:53.731193 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:53.731453 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:53.731454 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:53.731454 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:53.731617 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:53.731900 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:53.731906 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:53.732080 (+ 174us) client_negotiation.cc:769] Sending connection context
0929 10:50:53.732207 (+ 127us) client_negotiation.cc:240] Negotiation successful
0929 10:50:53.732337 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":62,"threads_started":1}
I20260929 10:50:53.733145 31305 heartbeater.cc:347] Connected to a master server at 127.20.80.254:46581
I20260929 10:50:53.733253 31305 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:53.733449 31305 heartbeater.cc:511] Master 127.20.80.254:46581 requested a full tablet report, sending...
I20260929 10:50:53.734033 31113 ts_manager.cc:194] Registered new tserver with Master: eb0dfb4ba76246b685d61ffd228456e3 (127.20.80.193:34685)
I20260929 10:50:53.734648 31113 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.193@KRBTEST.COM'} at 127.20.80.193:32783
WARNING: no policy specified for kudu/127.20.80.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
I20260929 10:50:53.759423 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.194:0
--local_ip_for_outbound_sockets=127.20.80.194
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:46581
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:43863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:53.840199 31311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:53.840399 31311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:53.840421 31311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:53.841588 31311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:53.841629 31311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:53.841655 31311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.194
I20260929 10:50:53.843173 31311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:43863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:46581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.194
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:53.843608 31311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:53.843902 31311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:53.848994 31316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:53.849023 31317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:53.849035 31319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:53.849113 31311 server_base.cc:1034] running on GCE node
I20260929 10:50:53.849380 31311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:53.849619 31311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:53.850809 31311 hybrid_clock.cc:648] HybridClock initialized: now 1790679053850786 us; error 54 us; skew 500 ppm
Sep 29 10:50:53 dist-test-slave-vqch krb5kdc[31068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679053, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:53.853307 31311 init.cc:377] Logged in from keytab as kudu/127.20.80.194@KRBTEST.COM (short username kudu)
I20260929 10:50:53.853727 31311 webserver.cc:460] Webserver started at http://127.20.80.194:33557/ using document root <none> and password file <none>
I20260929 10:50:53.853962 31311 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:53.854034 31311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:53.854183 31311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:53.855208 31311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/data/instance:
uuid: "9b323e88fb50473da54a22099d713e38"
format_stamp: "Formatted at 2026-09-29 10:50:53 on dist-test-slave-vqch"
server_key: "3048947802ac7be4d4b25ebf09b97250"
server_key_iv: "b15db33cc93361dd70886cf13edd49df"
server_key_version: "encryptionkey@0"
I20260929 10:50:53.855563 31311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance:
uuid: "9b323e88fb50473da54a22099d713e38"
format_stamp: "Formatted at 2026-09-29 10:50:53 on dist-test-slave-vqch"
server_key: "3048947802ac7be4d4b25ebf09b97250"
server_key_iv: "b15db33cc93361dd70886cf13edd49df"
server_key_version: "encryptionkey@0"
I20260929 10:50:53.856923 31311 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:50:53.857692 31326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:53.857861 31311 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:50:53.857959 31311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/wal
uuid: "9b323e88fb50473da54a22099d713e38"
format_stamp: "Formatted at 2026-09-29 10:50:53 on dist-test-slave-vqch"
server_key: "3048947802ac7be4d4b25ebf09b97250"
server_key_iv: "b15db33cc93361dd70886cf13edd49df"
server_key_version: "encryptionkey@0"
I20260929 10:50:53.858044 31311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:53.877887 31311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:53.879941 31311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:53.880098 31311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:53.880323 31311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:53.880630 31311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:53.880682 31311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:53.880724 31311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:53.880751 31311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:53.886495 31311 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.194:34667
I20260929 10:50:53.886538 31439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.194:34667 every 8 connection(s)
I20260929 10:50:53.886840 31311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
Sep 29 10:50:53 dist-test-slave-vqch krb5kdc[31068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679053, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:53.894054 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 31311
I20260929 10:50:53.894158 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance
I20260929 10:50:53.894353 20803 external_mini_cluster.cc:1428] Setting key 1a62be52288651cefe9874952393587a
I20260929 10:50:53.896497 31158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:53.887561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:48101 (local address 127.20.80.254:46581)
0929 10:50:53.887632 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:53.887634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:53.887837 (+ 203us) server_negotiation.cc:407] Connection header received
0929 10:50:53.888386 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:53.888388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:53.888401 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:53.888461 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:53.889461 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:53.889962 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:53.890936 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:53.891138 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:53.891225 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:53.893347 (+ 2122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:53.893348 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:53.893349 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:53.893355 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:53.894844 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:53.895364 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:53.895364 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:53.895365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:53.895455 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:53.895689 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:53.895690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:53.895690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:53.895843 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:53.895915 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:53.896249 (+ 334us) server_negotiation.cc:299] Negotiation successful
0929 10:50:53.896346 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 10:50:53.896564 31442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:53.887540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:46581 (local address 127.20.80.194:48101)
0929 10:50:53.887714 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:53.887756 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:53.888260 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:53.888536 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:53.888540 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:53.889179 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:53.889365 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:53.889371 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:53.890055 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:53.890056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:53.890836 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:53.890838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:53.891237 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:53.891237 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:53.891360 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:53.891804 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:53.891811 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:53.893258 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:53.894920 (+ 1662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:53.894927 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:53.894929 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:53.895279 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:53.895523 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:53.895523 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:53.895524 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:53.895623 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:53.895953 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:53.895959 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:53.896097 (+ 138us) client_negotiation.cc:769] Sending connection context
0929 10:50:53.896209 (+ 112us) client_negotiation.cc:240] Negotiation successful
0929 10:50:53.896344 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":80,"threads_started":1}
I20260929 10:50:53.897100 31440 heartbeater.cc:347] Connected to a master server at 127.20.80.254:46581
I20260929 10:50:53.897197 31440 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:53.897399 31440 heartbeater.cc:511] Master 127.20.80.254:46581 requested a full tablet report, sending...
I20260929 10:50:53.897774 31113 ts_manager.cc:194] Registered new tserver with Master: 9b323e88fb50473da54a22099d713e38 (127.20.80.194:34667)
I20260929 10:50:53.898249 31113 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.194@KRBTEST.COM'} at 127.20.80.194:48101
WARNING: no policy specified for kudu/127.20.80.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
I20260929 10:50:53.921324 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.195:0
--local_ip_for_outbound_sockets=127.20.80.195
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:46581
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:43863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:50:54.004611 31445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:50:54.004806 31445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:50:54.004829 31445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:50:54.006119 31445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:50:54.006179 31445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:50:54.006229 31445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.195
I20260929 10:50:54.007833 31445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:43863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:46581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.195
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:50:54.008074 31445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:50:54.008297 31445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:50:54.013231 31452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:50:54.013266 31450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:54.013367 31445 server_base.cc:1034] running on GCE node
W20260929 10:50:54.013388 31454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:50:54.013626 31445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:50:54.013927 31445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:50:54.015118 31445 hybrid_clock.cc:648] HybridClock initialized: now 1790679054015094 us; error 33 us; skew 500 ppm
Sep 29 10:50:54 dist-test-slave-vqch krb5kdc[31068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679054, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:50:54.017725 31445 init.cc:377] Logged in from keytab as kudu/127.20.80.195@KRBTEST.COM (short username kudu)
I20260929 10:50:54.018092 31445 webserver.cc:460] Webserver started at http://127.20.80.195:41943/ using document root <none> and password file <none>
I20260929 10:50:54.018263 31445 fs_manager.cc:359] Metadata directory not provided
I20260929 10:50:54.018302 31445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:50:54.018386 31445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:50:54.019343 31445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/data/instance:
uuid: "614dca1fb0a34f8faafc6e9398961edb"
format_stamp: "Formatted at 2026-09-29 10:50:54 on dist-test-slave-vqch"
server_key: "035ccd31edd335a0700a2214f6b5a945"
server_key_iv: "e17ccbe3d77c46ebd74fdab51e011ceb"
server_key_version: "encryptionkey@0"
I20260929 10:50:54.019639 31445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance:
uuid: "614dca1fb0a34f8faafc6e9398961edb"
format_stamp: "Formatted at 2026-09-29 10:50:54 on dist-test-slave-vqch"
server_key: "035ccd31edd335a0700a2214f6b5a945"
server_key_iv: "e17ccbe3d77c46ebd74fdab51e011ceb"
server_key_version: "encryptionkey@0"
I20260929 10:50:54.020823 31445 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 10:50:54.021621 31461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:54.021770 31445 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:50:54.021854 31445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/wal
uuid: "614dca1fb0a34f8faafc6e9398961edb"
format_stamp: "Formatted at 2026-09-29 10:50:54 on dist-test-slave-vqch"
server_key: "035ccd31edd335a0700a2214f6b5a945"
server_key_iv: "e17ccbe3d77c46ebd74fdab51e011ceb"
server_key_version: "encryptionkey@0"
I20260929 10:50:54.021908 31445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:50:54.030447 31445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:50:54.032771 31445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:50:54.032888 31445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:50:54.033145 31445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:50:54.033473 31445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:50:54.033507 31445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:54.033529 31445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:50:54.033576 31445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:50:54.039422 31445 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.195:44563
I20260929 10:50:54.039501 31574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.195:44563 every 8 connection(s)
I20260929 10:50:54.039839 31445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
Sep 29 10:50:54 dist-test-slave-vqch krb5kdc[31068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679054, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:50:54.045969 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 31445
I20260929 10:50:54.046089 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance
I20260929 10:50:54.046269 20803 external_mini_cluster.cc:1428] Setting key 2976e71bc7f91f8a5a20083edc9f836f
I20260929 10:50:54.049074 31158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:54.040629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:41815 (local address 127.20.80.254:46581)
0929 10:50:54.040693 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:54.040694 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:54.040916 (+ 222us) server_negotiation.cc:407] Connection header received
0929 10:50:54.041461 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:54.041462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:54.041473 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:54.041538 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:54.042473 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:54.042978 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:54.043940 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:54.044121 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:54.044176 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:54.046389 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:54.046391 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:54.046392 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:54.046399 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:54.047844 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:54.048223 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:54.048223 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:54.048224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:54.048273 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:54.048482 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:54.048483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:54.048483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:54.048580 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:54.048630 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:54.048878 (+ 248us) server_negotiation.cc:299] Negotiation successful
0929 10:50:54.048940 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 10:50:54.049221 31577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:54.040617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:46581 (local address 127.20.80.195:41815)
0929 10:50:54.040779 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:54.040821 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:54.041370 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:54.041596 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:54.041599 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:54.042184 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:50:54.042381 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:54.042386 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:54.043063 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:54.043063 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:54.043857 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:54.043859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:54.044195 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:54.044195 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:54.044291 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:54.044753 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:50:54.044761 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:50:54.046282 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:50:54.047926 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:54.047932 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:54.047934 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:54.048158 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:54.048332 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:50:54.048333 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:50:54.048333 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:50:54.048428 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:50:54.048634 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:50:54.048638 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:50:54.048768 (+ 130us) client_negotiation.cc:769] Sending connection context
0929 10:50:54.048851 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 10:50:54.048965 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":79,"threads_started":1}
I20260929 10:50:54.049691 31575 heartbeater.cc:347] Connected to a master server at 127.20.80.254:46581
I20260929 10:50:54.049829 31575 heartbeater.cc:464] Registering TS with master...
I20260929 10:50:54.050020 31575 heartbeater.cc:511] Master 127.20.80.254:46581 requested a full tablet report, sending...
I20260929 10:50:54.050403 31113 ts_manager.cc:194] Registered new tserver with Master: 614dca1fb0a34f8faafc6e9398961edb (127.20.80.195:44563)
I20260929 10:50:54.050802 31113 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.195@KRBTEST.COM'} at 127.20.80.195:41815
I20260929 10:50:54.058251 20803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:50:54.065845 31158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:54.059518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37138 (local address 127.20.80.254:46581)
0929 10:50:54.059577 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:54.059579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:54.059693 (+ 114us) server_negotiation.cc:407] Connection header received
0929 10:50:54.059781 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:54.059782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:54.059793 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:54.059849 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:50:54.060402 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:54.060961 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:54.061720 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:54.061938 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:54.062023 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:54.063185 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:50:54.063187 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:50:54.063187 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:50:54.063194 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:50:54.064553 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:54.065008 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:54.065009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:54.065010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:54.065072 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:50:54.065292 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:50:54.065293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:50:54.065293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:50:54.065451 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:50:54.065564 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:54.065664 (+ 100us) server_negotiation.cc:299] Negotiation successful
0929 10:50:54.065744 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 10:50:54.067463 31113 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37138:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:50:54.069330 31113 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:50:54.076164 31590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:54.071941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41164 (local address 127.20.80.195:44563)
0929 10:50:54.072295 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:54.072300 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:54.072310 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:50:54.072370 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:54.072372 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:54.072390 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:54.072483 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:54.072662 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:54.073816 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:54.075047 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:54.075726 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:54.075807 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:54.075866 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:54.076042 (+ 176us) server_negotiation.cc:299] Negotiation successful
0929 10:50:54.076080 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":92,"threads_started":1}
I20260929 10:50:54.076200 31586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:54.071714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:44563 (local address 127.0.0.1:41164)
0929 10:50:54.071885 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:54.071928 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:54.071996 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:54.072500 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:54.072504 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:54.072506 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:54.072574 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:54.072577 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:54.073920 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:54.073921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:54.074986 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:54.074988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:54.075832 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:54.075833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:54.075934 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:54.075938 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:50:54.076019 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 10:50:54.076103 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":8,"thread_start_us":111,"threads_started":1}
I20260929 10:50:54.076200 31589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:54.071757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:34685 (local address 127.0.0.1:34368)
0929 10:50:54.072122 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:54.072132 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:54.072181 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:54.072779 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:54.072780 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:54.072781 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:54.072818 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:54.072819 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:54.074003 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:54.074004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:54.074894 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:54.074897 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:54.075834 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:54.075834 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:54.075920 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:54.075937 (+ 17us) client_negotiation.cc:769] Sending connection context
0929 10:50:54.076019 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 10:50:54.076103 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":38,"thread_start_us":77,"threads_started":1}
I20260929 10:50:54.076862 31588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:54.071888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34368 (local address 127.20.80.193:34685)
0929 10:50:54.072619 (+ 731us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:54.072623 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:54.072633 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:50:54.072669 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:54.072670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:54.072684 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:54.072775 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:54.072891 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:54.073927 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:54.074991 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:54.075726 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:54.075810 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:54.075866 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:54.076711 (+ 845us) server_negotiation.cc:299] Negotiation successful
0929 10:50:54.076767 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":698,"thread_start_us":98,"threads_started":1}
I20260929 10:50:54.076949 31587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:54.071967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:34667 (local address 127.0.0.1:58394)
0929 10:50:54.072153 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:54.072161 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:54.072205 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:54.072555 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:54.072556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:54.072558 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:54.072609 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:54.072611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:54.073677 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:54.073678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:54.074968 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:54.074971 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:54.076732 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:54.076733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:54.076790 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:54.076795 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 10:50:54.076825 (+ 30us) client_negotiation.cc:240] Negotiation successful
0929 10:50:54.076867 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":58,"threads_started":1}
I20260929 10:50:54.077047 31509 tablet_service.cc:1467] Processing CreateTablet for tablet 77921477d8314aff9785d8d8a549ba63 (DEFAULT_TABLE table=test-table [id=d03e13e6056b445380781d51a607539e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:54.077376 31591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:54.072200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58394 (local address 127.20.80.194:34667)
0929 10:50:54.072345 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:54.072349 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:54.072356 (+ 7us) server_negotiation.cc:407] Connection header received
0929 10:50:54.072436 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:54.072438 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:54.072454 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:54.072685 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:54.072813 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:54.073578 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:54.075902 (+ 2324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:54.076630 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:54.077150 (+ 520us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:54.077198 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:54.077219 (+ 21us) server_negotiation.cc:299] Negotiation successful
0929 10:50:54.077292 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260929 10:50:54.077430 31509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77921477d8314aff9785d8d8a549ba63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:54.077631 31239 tablet_service.cc:1467] Processing CreateTablet for tablet 77921477d8314aff9785d8d8a549ba63 (DEFAULT_TABLE table=test-table [id=d03e13e6056b445380781d51a607539e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:54.077936 31239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77921477d8314aff9785d8d8a549ba63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:54.077975 31374 tablet_service.cc:1467] Processing CreateTablet for tablet 77921477d8314aff9785d8d8a549ba63 (DEFAULT_TABLE table=test-table [id=d03e13e6056b445380781d51a607539e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:50:54.078307 31374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77921477d8314aff9785d8d8a549ba63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:50:54.080626 31592 tablet_bootstrap.cc:492] T 77921477d8314aff9785d8d8a549ba63 P 9b323e88fb50473da54a22099d713e38: Bootstrap starting.
I20260929 10:50:54.081041 31593 tablet_bootstrap.cc:492] T 77921477d8314aff9785d8d8a549ba63 P 614dca1fb0a34f8faafc6e9398961edb: Bootstrap starting.
I20260929 10:50:54.081120 31594 tablet_bootstrap.cc:492] T 77921477d8314aff9785d8d8a549ba63 P eb0dfb4ba76246b685d61ffd228456e3: Bootstrap starting.
I20260929 10:50:54.081532 31592 tablet_bootstrap.cc:654] T 77921477d8314aff9785d8d8a549ba63 P 9b323e88fb50473da54a22099d713e38: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:54.081714 31593 tablet_bootstrap.cc:654] T 77921477d8314aff9785d8d8a549ba63 P 614dca1fb0a34f8faafc6e9398961edb: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:54.081753 31594 tablet_bootstrap.cc:654] T 77921477d8314aff9785d8d8a549ba63 P eb0dfb4ba76246b685d61ffd228456e3: Neither blocks nor log segments found. Creating new log.
I20260929 10:50:54.082141 31592 log.cc:824] T 77921477d8314aff9785d8d8a549ba63 P 9b323e88fb50473da54a22099d713e38: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:54.082203 31593 log.cc:824] T 77921477d8314aff9785d8d8a549ba63 P 614dca1fb0a34f8faafc6e9398961edb: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:54.082355 31594 log.cc:824] T 77921477d8314aff9785d8d8a549ba63 P eb0dfb4ba76246b685d61ffd228456e3: Log is configured to *not* fsync() on all Append() calls
I20260929 10:50:54.082947 31592 tablet_bootstrap.cc:492] T 77921477d8314aff9785d8d8a549ba63 P 9b323e88fb50473da54a22099d713e38: No bootstrap required, opened a new log
I20260929 10:50:54.083024 31592 ts_tablet_manager.cc:1397] T 77921477d8314aff9785d8d8a549ba63 P 9b323e88fb50473da54a22099d713e38: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260929 10:50:54.083086 31594 tablet_bootstrap.cc:492] T 77921477d8314aff9785d8d8a549ba63 P eb0dfb4ba76246b685d61ffd228456e3: No bootstrap required, opened a new log
I20260929 10:50:54.083158 31593 tablet_bootstrap.cc:492] T 77921477d8314aff9785d8d8a549ba63 P 614dca1fb0a34f8faafc6e9398961edb: No bootstrap required, opened a new log
I20260929 10:50:54.083158 31594 ts_tablet_manager.cc:1397] T 77921477d8314aff9785d8d8a549ba63 P eb0dfb4ba76246b685d61ffd228456e3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:50:54.083214 31593 ts_tablet_manager.cc:1397] T 77921477d8314aff9785d8d8a549ba63 P 614dca1fb0a34f8faafc6e9398961edb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:50:54.084409 31592 raft_consensus.cc:348] T 77921477d8314aff9785d8d8a549ba63 P 9b323e88fb50473da54a22099d713e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "614dca1fb0a34f8faafc6e9398961edb" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 44563 } } peers { permanent_uuid: "eb0dfb4ba76246b685d61ffd228456e3" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34685 } } peers { permanent_uuid: "9b323e88fb50473da54a22099d713e38" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34667 } }
I20260929 10:50:54.084561 31592 raft_consensus.cc:374] T 77921477d8314aff9785d8d8a549ba63 P 9b323e88fb50473da54a22099d713e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:54.084604 31592 raft_consensus.cc:729] T 77921477d8314aff9785d8d8a549ba63 P 9b323e88fb50473da54a22099d713e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b323e88fb50473da54a22099d713e38, State: Initialized, Role: FOLLOWER
I20260929 10:50:54.084709 31592 consensus_queue.cc:260] T 77921477d8314aff9785d8d8a549ba63 P 9b323e88fb50473da54a22099d713e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "614dca1fb0a34f8faafc6e9398961edb" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 44563 } } peers { permanent_uuid: "eb0dfb4ba76246b685d61ffd228456e3" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34685 } } peers { permanent_uuid: "9b323e88fb50473da54a22099d713e38" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34667 } }
I20260929 10:50:54.085036 31440 heartbeater.cc:503] Master 127.20.80.254:46581 was elected leader, sending a full tablet report...
I20260929 10:50:54.085078 31592 ts_tablet_manager.cc:1428] T 77921477d8314aff9785d8d8a549ba63 P 9b323e88fb50473da54a22099d713e38: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260929 10:50:54.085254 31594 raft_consensus.cc:348] T 77921477d8314aff9785d8d8a549ba63 P eb0dfb4ba76246b685d61ffd228456e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "614dca1fb0a34f8faafc6e9398961edb" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 44563 } } peers { permanent_uuid: "eb0dfb4ba76246b685d61ffd228456e3" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34685 } } peers { permanent_uuid: "9b323e88fb50473da54a22099d713e38" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34667 } }
I20260929 10:50:54.085413 31594 raft_consensus.cc:374] T 77921477d8314aff9785d8d8a549ba63 P eb0dfb4ba76246b685d61ffd228456e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:54.085453 31594 raft_consensus.cc:729] T 77921477d8314aff9785d8d8a549ba63 P eb0dfb4ba76246b685d61ffd228456e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0dfb4ba76246b685d61ffd228456e3, State: Initialized, Role: FOLLOWER
I20260929 10:50:54.085418 31593 raft_consensus.cc:348] T 77921477d8314aff9785d8d8a549ba63 P 614dca1fb0a34f8faafc6e9398961edb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "614dca1fb0a34f8faafc6e9398961edb" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 44563 } } peers { permanent_uuid: "eb0dfb4ba76246b685d61ffd228456e3" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34685 } } peers { permanent_uuid: "9b323e88fb50473da54a22099d713e38" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34667 } }
I20260929 10:50:54.085551 31593 raft_consensus.cc:374] T 77921477d8314aff9785d8d8a549ba63 P 614dca1fb0a34f8faafc6e9398961edb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:50:54.085582 31593 raft_consensus.cc:729] T 77921477d8314aff9785d8d8a549ba63 P 614dca1fb0a34f8faafc6e9398961edb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 614dca1fb0a34f8faafc6e9398961edb, State: Initialized, Role: FOLLOWER
I20260929 10:50:54.085569 31594 consensus_queue.cc:260] T 77921477d8314aff9785d8d8a549ba63 P eb0dfb4ba76246b685d61ffd228456e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "614dca1fb0a34f8faafc6e9398961edb" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 44563 } } peers { permanent_uuid: "eb0dfb4ba76246b685d61ffd228456e3" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34685 } } peers { permanent_uuid: "9b323e88fb50473da54a22099d713e38" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34667 } }
I20260929 10:50:54.085678 31593 consensus_queue.cc:260] T 77921477d8314aff9785d8d8a549ba63 P 614dca1fb0a34f8faafc6e9398961edb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "614dca1fb0a34f8faafc6e9398961edb" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 44563 } } peers { permanent_uuid: "eb0dfb4ba76246b685d61ffd228456e3" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34685 } } peers { permanent_uuid: "9b323e88fb50473da54a22099d713e38" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34667 } }
I20260929 10:50:54.085860 31594 ts_tablet_manager.cc:1428] T 77921477d8314aff9785d8d8a549ba63 P eb0dfb4ba76246b685d61ffd228456e3: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260929 10:50:54.085922 31593 ts_tablet_manager.cc:1428] T 77921477d8314aff9785d8d8a549ba63 P 614dca1fb0a34f8faafc6e9398961edb: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260929 10:50:54.085984 31305 heartbeater.cc:503] Master 127.20.80.254:46581 was elected leader, sending a full tablet report...
I20260929 10:50:54.086228 31575 heartbeater.cc:503] Master 127.20.80.254:46581 was elected leader, sending a full tablet report...
I20260929 10:50:54.098975 31600 raft_consensus.cc:482] T 77921477d8314aff9785d8d8a549ba63 P 614dca1fb0a34f8faafc6e9398961edb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:50:54.099123 31600 raft_consensus.cc:504] T 77921477d8314aff9785d8d8a549ba63 P 614dca1fb0a34f8faafc6e9398961edb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "614dca1fb0a34f8faafc6e9398961edb" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 44563 } } peers { permanent_uuid: "eb0dfb4ba76246b685d61ffd228456e3" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34685 } } peers { permanent_uuid: "9b323e88fb50473da54a22099d713e38" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34667 } }
I20260929 10:50:54.099467 31600 leader_election.cc:288] T 77921477d8314aff9785d8d8a549ba63 P 614dca1fb0a34f8faafc6e9398961edb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb0dfb4ba76246b685d61ffd228456e3 (127.20.80.193:34685), 9b323e88fb50473da54a22099d713e38 (127.20.80.194:34667)
I20260929 10:50:54.102952 31601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:54.099729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:34667 (local address 127.20.80.195:58079)
0929 10:50:54.099899 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:54.099908 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:54.099967 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:54.100103 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:54.100104 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:54.100105 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:54.100168 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:54.100169 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:54.101043 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:54.101044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:54.101903 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:54.101906 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:54.102698 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:54.102699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:54.102767 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:54.102770 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:50:54.102814 (+ 44us) client_negotiation.cc:240] Negotiation successful
0929 10:50:54.102843 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":1,"thread_start_us":99,"threads_started":1}
I20260929 10:50:54.102985 31591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:54.099789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:58079 (local address 127.20.80.194:34667)
0929 10:50:54.099831 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:54.099833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:54.099978 (+ 145us) server_negotiation.cc:407] Connection header received
0929 10:50:54.100009 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:54.100010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:54.100025 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:54.100067 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:54.100241 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:54.100951 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:54.101989 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:54.102570 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:54.102645 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:54.102653 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:54.102859 (+ 206us) server_negotiation.cc:299] Negotiation successful
0929 10:50:54.102911 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 10:50:54.103016 31588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:54.099735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:50089 (local address 127.20.80.193:34685)
0929 10:50:54.099779 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:54.099781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:54.099791 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:50:54.099858 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:54.099859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:54.099871 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:54.099926 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:50:54.100197 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:54.100968 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:54.102032 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:54.102570 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:54.102645 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:54.102653 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:54.102900 (+ 247us) server_negotiation.cc:299] Negotiation successful
0929 10:50:54.102941 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 10:50:54.103039 31577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:54.099623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:34685 (local address 127.20.80.195:50089)
0929 10:50:54.099696 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:54.099709 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:50:54.099786 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:50:54.099976 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:50:54.099977 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:50:54.099979 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:50:54.100103 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:54.100105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:54.101080 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:54.101081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:54.101905 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:50:54.101907 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:50:54.102704 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:50:54.102705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:50:54.102778 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:54.102782 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:50:54.102886 (+ 104us) client_negotiation.cc:240] Negotiation successful
0929 10:50:54.102929 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":73}
I20260929 10:50:54.103315 31394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77921477d8314aff9785d8d8a549ba63" candidate_uuid: "614dca1fb0a34f8faafc6e9398961edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b323e88fb50473da54a22099d713e38" is_pre_election: true
I20260929 10:50:54.103312 31259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77921477d8314aff9785d8d8a549ba63" candidate_uuid: "614dca1fb0a34f8faafc6e9398961edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0dfb4ba76246b685d61ffd228456e3" is_pre_election: true
I20260929 10:50:54.103483 31394 raft_consensus.cc:2454] T 77921477d8314aff9785d8d8a549ba63 P 9b323e88fb50473da54a22099d713e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 614dca1fb0a34f8faafc6e9398961edb in term 0.
I20260929 10:50:54.103502 31259 raft_consensus.cc:2454] T 77921477d8314aff9785d8d8a549ba63 P eb0dfb4ba76246b685d61ffd228456e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 614dca1fb0a34f8faafc6e9398961edb in term 0.
I20260929 10:50:54.103713 31465 leader_election.cc:302] T 77921477d8314aff9785d8d8a549ba63 P 614dca1fb0a34f8faafc6e9398961edb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 614dca1fb0a34f8faafc6e9398961edb, eb0dfb4ba76246b685d61ffd228456e3; no voters:
I20260929 10:50:54.103834 31600 raft_consensus.cc:2781] T 77921477d8314aff9785d8d8a549ba63 P 614dca1fb0a34f8faafc6e9398961edb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:50:54.103914 31600 raft_consensus.cc:482] T 77921477d8314aff9785d8d8a549ba63 P 614dca1fb0a34f8faafc6e9398961edb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:50:54.103948 31600 raft_consensus.cc:3037] T 77921477d8314aff9785d8d8a549ba63 P 614dca1fb0a34f8faafc6e9398961edb [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:54.104637 31600 raft_consensus.cc:504] T 77921477d8314aff9785d8d8a549ba63 P 614dca1fb0a34f8faafc6e9398961edb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "614dca1fb0a34f8faafc6e9398961edb" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 44563 } } peers { permanent_uuid: "eb0dfb4ba76246b685d61ffd228456e3" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34685 } } peers { permanent_uuid: "9b323e88fb50473da54a22099d713e38" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34667 } }
I20260929 10:50:54.104770 31600 leader_election.cc:288] T 77921477d8314aff9785d8d8a549ba63 P 614dca1fb0a34f8faafc6e9398961edb [CANDIDATE]: Term 1 election: Requested vote from peers eb0dfb4ba76246b685d61ffd228456e3 (127.20.80.193:34685), 9b323e88fb50473da54a22099d713e38 (127.20.80.194:34667)
I20260929 10:50:54.104934 31259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77921477d8314aff9785d8d8a549ba63" candidate_uuid: "614dca1fb0a34f8faafc6e9398961edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0dfb4ba76246b685d61ffd228456e3"
I20260929 10:50:54.105028 31259 raft_consensus.cc:3037] T 77921477d8314aff9785d8d8a549ba63 P eb0dfb4ba76246b685d61ffd228456e3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:54.105059 31394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77921477d8314aff9785d8d8a549ba63" candidate_uuid: "614dca1fb0a34f8faafc6e9398961edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b323e88fb50473da54a22099d713e38"
I20260929 10:50:54.105156 31394 raft_consensus.cc:3037] T 77921477d8314aff9785d8d8a549ba63 P 9b323e88fb50473da54a22099d713e38 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:50:54.105729 31259 raft_consensus.cc:2454] T 77921477d8314aff9785d8d8a549ba63 P eb0dfb4ba76246b685d61ffd228456e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 614dca1fb0a34f8faafc6e9398961edb in term 1.
I20260929 10:50:54.105958 31465 leader_election.cc:302] T 77921477d8314aff9785d8d8a549ba63 P 614dca1fb0a34f8faafc6e9398961edb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 614dca1fb0a34f8faafc6e9398961edb, eb0dfb4ba76246b685d61ffd228456e3; no voters:
I20260929 10:50:54.105978 31394 raft_consensus.cc:2454] T 77921477d8314aff9785d8d8a549ba63 P 9b323e88fb50473da54a22099d713e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 614dca1fb0a34f8faafc6e9398961edb in term 1.
I20260929 10:50:54.106055 31600 raft_consensus.cc:2781] T 77921477d8314aff9785d8d8a549ba63 P 614dca1fb0a34f8faafc6e9398961edb [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:50:54.106222 31600 raft_consensus.cc:686] T 77921477d8314aff9785d8d8a549ba63 P 614dca1fb0a34f8faafc6e9398961edb [term 1 LEADER]: Becoming Leader. State: Replica: 614dca1fb0a34f8faafc6e9398961edb, State: Running, Role: LEADER
I20260929 10:50:54.106334 31600 consensus_queue.cc:237] T 77921477d8314aff9785d8d8a549ba63 P 614dca1fb0a34f8faafc6e9398961edb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "614dca1fb0a34f8faafc6e9398961edb" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 44563 } } peers { permanent_uuid: "eb0dfb4ba76246b685d61ffd228456e3" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34685 } } peers { permanent_uuid: "9b323e88fb50473da54a22099d713e38" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34667 } }
I20260929 10:50:54.107163 31113 catalog_manager.cc:5495] T 77921477d8314aff9785d8d8a549ba63 P 614dca1fb0a34f8faafc6e9398961edb reported cstate change: term changed from 0 to 1, leader changed from <none> to 614dca1fb0a34f8faafc6e9398961edb (127.20.80.195). New cstate: current_term: 1 leader_uuid: "614dca1fb0a34f8faafc6e9398961edb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "614dca1fb0a34f8faafc6e9398961edb" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 44563 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb0dfb4ba76246b685d61ffd228456e3" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b323e88fb50473da54a22099d713e38" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34667 } health_report { overall_health: UNKNOWN } } }
I20260929 10:50:54.111931 31590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:54.109844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41178 (local address 127.20.80.195:44563)
0929 10:50:54.109879 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0929 10:50:54.109881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:50:54.109888 (+ 7us) server_negotiation.cc:407] Connection header received
0929 10:50:54.109916 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:50:54.109917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:50:54.109939 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:50:54.109997 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:50:54.110163 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:54.110607 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:54.111310 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:50:54.111486 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:50:54.111540 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:50:54.111669 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:50:54.111763 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:50:54.111823 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 10:50:54.111855 (+ 32us) server_negotiation.cc:299] Negotiation successful
0929 10:50:54.111871 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 10:50:54.113487 31259 raft_consensus.cc:1261] T 77921477d8314aff9785d8d8a549ba63 P eb0dfb4ba76246b685d61ffd228456e3 [term 1 FOLLOWER]: Refusing update from remote peer 614dca1fb0a34f8faafc6e9398961edb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:54.113559 31394 raft_consensus.cc:1261] T 77921477d8314aff9785d8d8a549ba63 P 9b323e88fb50473da54a22099d713e38 [term 1 FOLLOWER]: Refusing update from remote peer 614dca1fb0a34f8faafc6e9398961edb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:50:54.113818 31600 consensus_queue.cc:1035] T 77921477d8314aff9785d8d8a549ba63 P 614dca1fb0a34f8faafc6e9398961edb [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb0dfb4ba76246b685d61ffd228456e3" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 34685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:50:54.113971 31602 consensus_queue.cc:1035] T 77921477d8314aff9785d8d8a549ba63 P 614dca1fb0a34f8faafc6e9398961edb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b323e88fb50473da54a22099d713e38" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:50:54.116370 31606 mvcc.cc:205] Tried to move back new op lower bound from 7334621405646610432 to 7334621405620232192. Current Snapshot: MvccSnapshot[applied={T|T < 7334621405646610432}]
I20260929 10:50:54.116787 31607 mvcc.cc:205] Tried to move back new op lower bound from 7334621405646610432 to 7334621405620232192. Current Snapshot: MvccSnapshot[applied={T|T < 7334621405646610432}]
I20260929 10:50:54.118369 31113 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:50:54.118453 31113 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:50:54.119179 31113 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c50d94a7bcb9489d80a93e7f466a93fc: Sending DeleteTablet for 3 replicas of tablet 77921477d8314aff9785d8d8a549ba63
I20260929 10:50:54.119508 31509 tablet_service.cc:1514] Processing DeleteTablet for tablet 77921477d8314aff9785d8d8a549ba63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:54 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:41164
I20260929 10:50:54.119541 31374 tablet_service.cc:1514] Processing DeleteTablet for tablet 77921477d8314aff9785d8d8a549ba63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:54 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:58394
I20260929 10:50:54.119568 31239 tablet_service.cc:1514] Processing DeleteTablet for tablet 77921477d8314aff9785d8d8a549ba63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:50:54 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:34368
I20260929 10:50:54.119757 31617 tablet_replica.cc:330] T 77921477d8314aff9785d8d8a549ba63 P 9b323e88fb50473da54a22099d713e38: stopping tablet replica
I20260929 10:50:54.119853 31617 raft_consensus.cc:2229] T 77921477d8314aff9785d8d8a549ba63 P 9b323e88fb50473da54a22099d713e38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:50:54.119957 31617 raft_consensus.cc:2258] T 77921477d8314aff9785d8d8a549ba63 P 9b323e88fb50473da54a22099d713e38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:54.120101 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 31176
I20260929 10:50:54.120115 31615 tablet_replica.cc:330] T 77921477d8314aff9785d8d8a549ba63 P 614dca1fb0a34f8faafc6e9398961edb: stopping tablet replica
I20260929 10:50:54.121804 31615 raft_consensus.cc:2229] T 77921477d8314aff9785d8d8a549ba63 P 614dca1fb0a34f8faafc6e9398961edb [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:50:54.121928 31615 raft_consensus.cc:2258] T 77921477d8314aff9785d8d8a549ba63 P 614dca1fb0a34f8faafc6e9398961edb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:50:54.122393 31615 ts_tablet_manager.cc:1905] T 77921477d8314aff9785d8d8a549ba63 P 614dca1fb0a34f8faafc6e9398961edb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:54.124365 31615 ts_tablet_manager.cc:1918] T 77921477d8314aff9785d8d8a549ba63 P 614dca1fb0a34f8faafc6e9398961edb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:50:54.124439 31615 log.cc:1196] T 77921477d8314aff9785d8d8a549ba63 P 614dca1fb0a34f8faafc6e9398961edb: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790678996126930-20803-0/minicluster-data/ts-2/wal/wals/77921477d8314aff9785d8d8a549ba63
I20260929 10:50:54.124765 31615 ts_tablet_manager.cc:1939] T 77921477d8314aff9785d8d8a549ba63 P 614dca1fb0a34f8faafc6e9398961edb: Deleting consensus metadata
W20260929 10:50:54.126264 31100 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.80.193:34685 (error 108)
I20260929 10:50:54.126474 31097 catalog_manager.cc:4841] TS 614dca1fb0a34f8faafc6e9398961edb (127.20.80.195:44563): tablet 77921477d8314aff9785d8d8a549ba63 (table test-table [id=d03e13e6056b445380781d51a607539e]) successfully deleted
I20260929 10:50:54.126713 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 31311
I20260929 10:50:54.126965 31617 ts_tablet_manager.cc:1905] T 77921477d8314aff9785d8d8a549ba63 P 9b323e88fb50473da54a22099d713e38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:50:54.132210 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 31445
I20260929 10:50:54.132366 31587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:54.132216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:34685 (local address 127.0.0.1:34370)
0929 10:50:54.132275 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:54.132324 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.193:34685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260929 10:50:54.132515 31100 catalog_manager.cc:4584] TS eb0dfb4ba76246b685d61ffd228456e3 (127.20.80.193:34685): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 77921477d8314aff9785d8d8a549ba63: Network error: Client connection negotiation failed: client connection to 127.20.80.193:34685: connect: Connection refused (error 111)
I20260929 10:50:54.133409 31587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:50:54.132441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:34667 (local address 127.0.0.1:58404)
0929 10:50:54.133347 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0929 10:50:54.133376 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.194:34667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":889}
I20260929 10:50:54.136588 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 31081
2026-09-29T10:50:54Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1886 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:50:54 dist-test-slave-vqch krb5kdc[31621](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:50:54 dist-test-slave-vqch krb5kdc[31621](info): set up 2 sockets
Sep 29 10:50:54 dist-test-slave-vqch krb5kdc[31621](info): commencing operation
krb5kdc: starting...
W20260929 10:50:55.173548 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:50:55 dist-test-slave-vqch krb5kdc[31621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:50:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:50:55Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.20.80.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.20.80.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.20.80.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/rangeradmin_127.20.80.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.20.80.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/rangeradmin_127.20.80.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.20.80.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.20.80.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.20.80.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/rangerlookup_127.20.80.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.20.80.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/rangerlookup_127.20.80.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.80.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.80.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.80.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/HTTP_127.20.80.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.80.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/HTTP_127.20.80.212@KRBTEST.COM.keytab.
I20260929 10:50:55.339422 20803 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskfQc4Oq/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/postgres -l logfile start
2026-09-29 10:50:57.940 UTC [31651] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-29 10:50:57.940 UTC [31651] LOG: listening on IPv4 address "127.20.80.212", port 39907
2026-09-29 10:50:57.944 UTC [31651] LOG: listening on Unix socket "/tmp/.s.PGSQL.39907"
2026-09-29 10:50:57.965 UTC [31654] LOG: database system was shut down at 2026-09-29 10:50:56 UTC
2026-09-29 10:50:57.968 UTC [31651] LOG: database system is ready to accept connections
I20260929 10:50:58.929603 20803 mini_postgres.cc:96] Postgres bound to 39907
2026-09-29 10:50:58.935 UTC [31662] FATAL: database "slave" does not exist
127.20.80.212:39907 - accepting connections
I20260929 10:50:58.936452 20803 mini_ranger.cc:161] Starting Ranger...
I20260929 10:50:58.951557 20803 mini_ranger.cc:84] Created miniranger Postgres user
I20260929 10:50:59.469679 20803 mini_ranger.cc:87] Created ranger Postgres database
I20260929 10:50:59.469767 20803 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ranger-admin
2026-09-29 10:50:59,803 [I] DB FLAVOR :POSTGRES
2026-09-29 10:50:59,804 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 10:50:59,804 [I] Checking connection..
2026-09-29 10:50:59,804 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:39907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-29 10:51:00,056 [I] Checking connection passed.
2026-09-29 10:51:00,057 [I] --------- Verifying version history table ---------
2026-09-29 10:51:00,057 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:39907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 10:51:00,305 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 10:51:00,306 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:39907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 10:51:00,556 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 10:51:00,557 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-29 10:51:00,557 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:39907/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-29 10:51:00.819 UTC [31727] WARNING: there is no transaction in progress
2026-09-29 10:51:00,827 [I] create_dbversion_catalog.sql file imported successfully
2026-09-29 10:51:00,827 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:39907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 10:51:01,082 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-29 10:51:01,082 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 10:51:01,083 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:39907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-29 10:51:01,338 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:39907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-29 10:51:01,580 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:39907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-vqch.c.gcp-upstream.internal','N') ;"
2026-09-29 10:51:01,809 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:39907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 10:51:02,053 [I] Table x_portal_user does not exist in database ranger
2026-09-29 10:51:02,054 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:39907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 10:51:02,299 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-29 10:51:02,300 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:39907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 10:51:02,550 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:39907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 10:51:02,811 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-29 10:51:02,812 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:39907/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-29 10:51:03.821 UTC [31837] WARNING: there is no transaction in progress
2026-09-29 10:51:03.837 UTC [31837] WARNING: there is no transaction in progress
2026-09-29 10:51:03.854 UTC [31837] WARNING: there is no transaction in progress
2026-09-29 10:51:03.869 UTC [31837] WARNING: there is no transaction in progress
2026-09-29 10:51:03.884 UTC [31837] WARNING: there is no transaction in progress
2026-09-29 10:51:03.901 UTC [31837] WARNING: there is no transaction in progress
2026-09-29 10:51:03.967 UTC [31837] WARNING: there is no transaction in progress
2026-09-29 10:51:03.974 UTC [31837] WARNING: there is no transaction in progress
2026-09-29 10:51:03.987 UTC [31837] WARNING: there is no transaction in progress
2026-09-29 10:51:03.997 UTC [31837] WARNING: there is no transaction in progress
2026-09-29 10:51:04.008 UTC [31837] WARNING: there is no transaction in progress
2026-09-29 10:51:04.022 UTC [31837] WARNING: there is no transaction in progress
2026-09-29 10:51:04.032 UTC [31837] WARNING: there is no transaction in progress
2026-09-29 10:51:04.038 UTC [31837] WARNING: there is no transaction in progress
2026-09-29 10:51:04.044 UTC [31837] WARNING: there is no transaction in progress
2026-09-29 10:51:04.051 UTC [31837] WARNING: there is no transaction in progress
2026-09-29 10:51:04,732 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-29 10:51:04,733 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:39907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-29 10:51:04,982 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-29 10:51:04,983 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:39907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 10:51:05,240 [I] Table x_portal_user already exists in database 'ranger'
2026-09-29 10:51:05,241 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:39907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 10:51:05,497 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-29 10:51:05,498 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:39907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 10:51:05,760 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:39907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 10:51:06,006 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:39907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-vqch.c.gcp-upstream.internal';"
2026-09-29 10:51:06,251 [I] CORE_DB_SCHEMA import status has been updated
2026-09-29 10:51:06,252 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:39907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-29 10:51:06,508 [I] DB_PATCHES have already been applied
I20260929 10:51:06.514228 20803 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ranger-admin:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskfQc4Oq/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260929 10:51:06.514312 20803 mini_ranger.cc:193] Using host: 127.20.80.212
I20260929 10:51:06.517331 20803 mini_ranger.cc:227] Ranger admin URL: http://127.20.80.212:59465
Sep 29, 2026 10:51:06 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 10:51:06 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 10:51:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ranger-admin/ews/webapp
Sep 29, 2026 10:51:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 29, 2026 10:51:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 29, 2026 10:51:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ranger-admin/ews/webapp].
Sep 29, 2026 10:51:07 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260929 10:51:07.621130 20803 mini_ranger.cc:160] Time spent starting Ranger: real 8.685s user 0.000s sys 0.009s
Sep 29, 2026 10:51:28 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 29, 2026 10:51:28 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Sep 29, 2026 10:51:28 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 29, 2026 10:51:28 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 29, 2026 10:51:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 29, 2026 10:51:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 29, 2026 10:51:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 29, 2026 10:51:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 29, 2026 10:51:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 29, 2026 10:51:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 29, 2026 10:51:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 29, 2026 10:51:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 29, 2026 10:51:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 29, 2026 10:51:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 29, 2026 10:51:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 29, 2026 10:51:28 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 29, 2026 10:51:28 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 29, 2026 10:51:29 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260929 10:51:29.949560 20803 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.20.80.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.20.80.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.20.80.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/rangerkms_127.20.80.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.20.80.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/rangerkms_127.20.80.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.80.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.80.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.80.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/HTTP_127.20.80.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.80.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/HTTP_127.20.80.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 29 10:51:30 dist-test-slave-vqch krb5kdc[31621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679090, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260929 10:51:30.035136 20803 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260929 10:51:30.043867 20803 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260929 10:51:31.004685 20803 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260929 10:51:31.004770 20803 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ranger-kms
I20260929 10:51:31.004792 20803 mini_ranger_kms.cc:227] Using postgres at 127.20.80.212:39907
2026-09-29 10:51:31,415 [I] DB FLAVOR :POSTGRES
2026-09-29 10:51:31,416 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 10:51:31,416 [I] Checking connection
2026-09-29 10:51:31,416 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:39907/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-29 10:51:31,661 [I] connection success
2026-09-29 10:51:31,661 [I] --------- Verifying Ranger DB tables ---------
2026-09-29 10:51:31,661 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-29 10:51:31,661 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:39907/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-29 10:51:31,892 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-29 10:51:31,893 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 10:51:31,893 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-29 10:51:31,893 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:39907/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-29 10:51:32,157 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260929 10:51:32.631868 20803 mini_ranger_kms.cc:310] Created kms service
I20260929 10:51:32.810338 20803 mini_ranger_kms.cc:326] Created kudu user
I20260929 10:51:32.856755 20803 mini_ranger_kms.cc:343] Created rangerkms user
Sep 29 10:51:32 dist-test-slave-vqch krb5kdc[31621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679090, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.20.80.212@KRBTEST.COM
I20260929 10:51:33.236157 20803 mini_ranger_kms.cc:384] Added ranger policy
I20260929 10:51:33.236284 20803 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ranger-kms:/tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260929 10:51:33.236307 20803 mini_ranger_kms.cc:240] Using host: 127.20.80.212
I20260929 10:51:33.240398 20803 mini_ranger_kms.cc:276] Ranger KMS PID: 32018
I20260929 10:51:33.240522 20803 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.20.80.212:41997
Sep 29, 2026 10:51:34 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 10:51:34 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 10:51:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 29, 2026 10:51:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 29, 2026 10:51:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ranger-kms/ews/webapp].
Sep 29, 2026 10:51:34 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260929 10:51:35.494023 20803 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.459s user 0.004s sys 0.201s
I20260929 10:51:35.494130 20803 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260929 10:51:35.494208 20803 mini_ranger_kms.cc:401] 127.20.80.212:41997/kms/v1/keys
Sep 29 10:51:35 dist-test-slave-vqch krb5kdc[31621](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790679095, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.80.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 29 10:51:36 dist-test-slave-vqch krb5kdc[31621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679095, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.80.212@KRBTEST.COM for HTTP/127.20.80.212@KRBTEST.COM
Sep 29 10:51:36 dist-test-slave-vqch krb5kdc[31621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679095, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.80.212@KRBTEST.COM for HTTP/127.20.80.212@KRBTEST.COM
Sep 29 10:51:38 dist-test-slave-vqch krb5kdc[31621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679098, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.20.80.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
I20260929 10:51:38.785454 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:32901
--webserver_interface=127.20.80.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:33851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:32901
--ranger_config_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.80.212:41997/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:51:38.867164 32070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:51:38.867419 32070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:51:38.867453 32070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:51:38.868635 32070 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:51:38.868705 32070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:51:38.868728 32070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:51:38.868789 32070 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:51:38.868820 32070 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:51:38.870425 32070 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:33851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/master-0/wal
--ranger_kms_url=127.20.80.212:41997/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:32901
--ranger_config_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:32901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:51:38.870709 32070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:51:38.870934 32070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 10:51:38.877240 32070 server_base.cc:1034] running on GCE node
W20260929 10:51:38.877213 32076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:51:38.877223 32075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:51:38.877717 32078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:51:38.878010 32070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:51:38.878300 32070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:51:38.879477 32070 hybrid_clock.cc:648] HybridClock initialized: now 1790679098879451 us; error 41 us; skew 500 ppm
Sep 29 10:51:38 dist-test-slave-vqch krb5kdc[31621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679098, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:51:38.886651 32070 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:51:38.887244 32070 webserver.cc:460] Webserver started at http://127.20.80.254:33625/ using document root <none> and password file <none>
I20260929 10:51:38.887471 32070 fs_manager.cc:359] Metadata directory not provided
I20260929 10:51:38.887529 32070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:51:38.887674 32070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 10:51:38 dist-test-slave-vqch krb5kdc[31621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679098, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for HTTP/127.20.80.212@KRBTEST.COM
I20260929 10:51:38.945231 32070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "53b931aec02a4231b3c8722955953981"
format_stamp: "Formatted at 2026-09-29 10:51:38 on dist-test-slave-vqch"
server_key: "b8af2eb97439bec6ba7ae53a757e3dbd"
server_key_iv: "8c463edab6b4375a5cacea586fbef0f1"
server_key_version: "kuduclusterkey@0"
I20260929 10:51:38.945677 32070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "53b931aec02a4231b3c8722955953981"
format_stamp: "Formatted at 2026-09-29 10:51:38 on dist-test-slave-vqch"
server_key: "b8af2eb97439bec6ba7ae53a757e3dbd"
server_key_iv: "8c463edab6b4375a5cacea586fbef0f1"
server_key_version: "kuduclusterkey@0"
I20260929 10:51:38.947642 32070 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260929 10:51:38.960812 32085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:51:38.961076 32070 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 10:51:38.961184 32070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "53b931aec02a4231b3c8722955953981"
format_stamp: "Formatted at 2026-09-29 10:51:38 on dist-test-slave-vqch"
server_key: "b8af2eb97439bec6ba7ae53a757e3dbd"
server_key_iv: "8c463edab6b4375a5cacea586fbef0f1"
server_key_version: "kuduclusterkey@0"
I20260929 10:51:38.961316 32070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:51:38.976013 32070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:51:38.978016 32070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:51:38.978186 32070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:51:38.983119 32070 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:32901
I20260929 10:51:38.983141 32137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:32901 every 8 connection(s)
I20260929 10:51:38.983512 32070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:51:38.984241 32138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:51:38.986639 32138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53b931aec02a4231b3c8722955953981: Bootstrap starting.
I20260929 10:51:38.987211 32138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53b931aec02a4231b3c8722955953981: Neither blocks nor log segments found. Creating new log.
I20260929 10:51:38.987517 32138 log.cc:824] T 00000000000000000000000000000000 P 53b931aec02a4231b3c8722955953981: Log is configured to *not* fsync() on all Append() calls
I20260929 10:51:38.988209 32138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53b931aec02a4231b3c8722955953981: No bootstrap required, opened a new log
I20260929 10:51:38.990321 32138 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53b931aec02a4231b3c8722955953981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b931aec02a4231b3c8722955953981" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 32901 } }
I20260929 10:51:38.990480 32138 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53b931aec02a4231b3c8722955953981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:51:38.990504 32138 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53b931aec02a4231b3c8722955953981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53b931aec02a4231b3c8722955953981, State: Initialized, Role: FOLLOWER
I20260929 10:51:38.990656 32138 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53b931aec02a4231b3c8722955953981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b931aec02a4231b3c8722955953981" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 32901 } }
I20260929 10:51:38.990728 32138 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53b931aec02a4231b3c8722955953981 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:51:38.990749 32138 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53b931aec02a4231b3c8722955953981 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:51:38.990815 32138 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53b931aec02a4231b3c8722955953981 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:51:38.991335 32138 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53b931aec02a4231b3c8722955953981 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b931aec02a4231b3c8722955953981" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 32901 } }
I20260929 10:51:38.991427 32138 leader_election.cc:302] T 00000000000000000000000000000000 P 53b931aec02a4231b3c8722955953981 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53b931aec02a4231b3c8722955953981; no voters:
I20260929 10:51:38.991641 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 32070
I20260929 10:51:38.991686 32138 leader_election.cc:288] T 00000000000000000000000000000000 P 53b931aec02a4231b3c8722955953981 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:51:38.991714 32141 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53b931aec02a4231b3c8722955953981 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:51:38.991747 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:51:38.991840 32141 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53b931aec02a4231b3c8722955953981 [term 1 LEADER]: Becoming Leader. State: Replica: 53b931aec02a4231b3c8722955953981, State: Running, Role: LEADER
I20260929 10:51:38.991921 32141 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53b931aec02a4231b3c8722955953981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b931aec02a4231b3c8722955953981" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 32901 } }
I20260929 10:51:38.991948 32138 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53b931aec02a4231b3c8722955953981 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:51:38.991954 20803 external_mini_cluster.cc:1428] Setting key 928504935e1394ec9050cf105f541797
I20260929 10:51:38.992844 32141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53b931aec02a4231b3c8722955953981 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53b931aec02a4231b3c8722955953981. Latest consensus state: current_term: 1 leader_uuid: "53b931aec02a4231b3c8722955953981" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b931aec02a4231b3c8722955953981" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 32901 } } }
I20260929 10:51:38.992857 32143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53b931aec02a4231b3c8722955953981 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53b931aec02a4231b3c8722955953981" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b931aec02a4231b3c8722955953981" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 32901 } } }
I20260929 10:51:38.993171 32143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53b931aec02a4231b3c8722955953981 [sys.catalog]: This master's current role is: LEADER
I20260929 10:51:38.993080 32141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53b931aec02a4231b3c8722955953981 [sys.catalog]: This master's current role is: LEADER
I20260929 10:51:38.993429 32138 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260929 10:51:38.996191 32157 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:51:38.997277 32157 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:51:38.999818 32157 catalog_manager.cc:1347] Generated new cluster ID: 0d81d092aec24dd9867b17a6629c2a4a
I20260929 10:51:38.999866 32157 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 10:51:39 dist-test-slave-vqch krb5kdc[31621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679098, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:51:39.011638 32150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:51:38.992626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54056 (local address 127.20.80.254:32901)
0929 10:51:38.993026 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0929 10:51:38.993035 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:51:38.993397 (+ 362us) server_negotiation.cc:407] Connection header received
0929 10:51:38.993526 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:51:38.993534 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:51:38.993602 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:51:38.993749 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:51:38.995473 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:51:38.998026 (+ 2553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:51:39.001047 (+ 3021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:51:39.001666 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:51:39.004825 (+ 3159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:51:39.007509 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:51:39.007514 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:51:39.007514 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:51:39.007527 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:51:39.009598 (+ 2071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:51:39.010003 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:51:39.010006 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:51:39.010007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:51:39.010104 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:51:39.010296 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:51:39.010297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:51:39.010297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:51:39.010605 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:51:39.011199 (+ 594us) server_negotiation.cc:1035] Waiting for connection context
0929 10:51:39.011274 (+ 75us) server_negotiation.cc:299] Negotiation successful
0929 10:51:39.011398 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":77,"threads_started":1}
I20260929 10:51:39.051442 32157 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:51:39.052261 32157 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:51:39.060773 32157 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53b931aec02a4231b3c8722955953981: Generated new TSK 0
I20260929 10:51:39.061026 32157 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.80.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
I20260929 10:51:39.777541 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.80.212:41997/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.193:0
--local_ip_for_outbound_sockets=127.20.80.193
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:32901
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:33851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:51:39.868122 32184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:51:39.868358 32184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:51:39.868397 32184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:51:39.869719 32184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:51:39.869788 32184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:51:39.869849 32184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.193
I20260929 10:51:39.871403 32184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:33851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.20.80.212:41997/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:32901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.193
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:51:39.871692 32184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:51:39.871939 32184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:51:39.878221 32189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:51:39.878353 32192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:51:39.878496 32184 server_base.cc:1034] running on GCE node
W20260929 10:51:39.878679 32190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:51:39.878953 32184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:51:39.879236 32184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:51:39.880419 32184 hybrid_clock.cc:648] HybridClock initialized: now 1790679099880386 us; error 45 us; skew 500 ppm
Sep 29 10:51:39 dist-test-slave-vqch krb5kdc[31621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679099, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:51:39.883718 32184 init.cc:377] Logged in from keytab as kudu/127.20.80.193@KRBTEST.COM (short username kudu)
I20260929 10:51:39.884315 32184 webserver.cc:460] Webserver started at http://127.20.80.193:40621/ using document root <none> and password file <none>
I20260929 10:51:39.884604 32184 fs_manager.cc:359] Metadata directory not provided
I20260929 10:51:39.884671 32184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:51:39.884822 32184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 10:51:39 dist-test-slave-vqch krb5kdc[31621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679099, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for HTTP/127.20.80.212@KRBTEST.COM
I20260929 10:51:39.899636 32184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-0/data/instance:
uuid: "629e6ed702cd4fe7a7d347b971737719"
format_stamp: "Formatted at 2026-09-29 10:51:39 on dist-test-slave-vqch"
server_key: "aa846408b9b49a6e7afcd508d15ad58d"
server_key_iv: "9c47df91663954c213bfa5924d615b2c"
server_key_version: "kuduclusterkey@0"
I20260929 10:51:39.900195 32184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance:
uuid: "629e6ed702cd4fe7a7d347b971737719"
format_stamp: "Formatted at 2026-09-29 10:51:39 on dist-test-slave-vqch"
server_key: "aa846408b9b49a6e7afcd508d15ad58d"
server_key_iv: "9c47df91663954c213bfa5924d615b2c"
server_key_version: "kuduclusterkey@0"
I20260929 10:51:39.902297 32184 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260929 10:51:39.917114 32199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:51:39.917351 32184 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:51:39.917447 32184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-0/wal
uuid: "629e6ed702cd4fe7a7d347b971737719"
format_stamp: "Formatted at 2026-09-29 10:51:39 on dist-test-slave-vqch"
server_key: "aa846408b9b49a6e7afcd508d15ad58d"
server_key_iv: "9c47df91663954c213bfa5924d615b2c"
server_key_version: "kuduclusterkey@0"
I20260929 10:51:39.917544 32184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:51:39.954927 32184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:51:39.957471 32184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:51:39.957638 32184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:51:39.957911 32184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:51:39.958300 32184 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:51:39.958352 32184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:51:39.958420 32184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:51:39.958465 32184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:51:39.965739 32184 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.193:38781
I20260929 10:51:39.966207 32184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
I20260929 10:51:39.966802 32315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.193:38781 every 8 connection(s)
Sep 29 10:51:39 dist-test-slave-vqch krb5kdc[31621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679099, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:51:39.976416 32318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:51:39.966949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:32901 (local address 127.20.80.193:43051)
0929 10:51:39.967194 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0929 10:51:39.967238 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 10:51:39.967761 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:51:39.968026 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:51:39.968031 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:51:39.968225 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:51:39.968428 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:51:39.968433 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:51:39.969204 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:51:39.969205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:51:39.970012 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:51:39.970014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:51:39.970426 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:51:39.970427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:51:39.970577 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:51:39.971218 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:51:39.971229 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:51:39.972897 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:51:39.974619 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:51:39.974625 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:51:39.974629 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:51:39.974962 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:51:39.975214 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:51:39.975214 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:51:39.975215 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:51:39.975366 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:51:39.975690 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:51:39.975697 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:51:39.975880 (+ 183us) client_negotiation.cc:769] Sending connection context
0929 10:51:39.976007 (+ 127us) client_negotiation.cc:240] Negotiation successful
0929 10:51:39.976177 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":84,"threads_started":1}
I20260929 10:51:39.976444 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 32184
I20260929 10:51:39.976540 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance
I20260929 10:51:39.976737 20803 external_mini_cluster.cc:1428] Setting key 80ae4e22939eb04450d6ff22fb70ffa7
I20260929 10:51:39.979015 32319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:51:39.966969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:43051 (local address 127.20.80.254:32901)
0929 10:51:39.967135 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0929 10:51:39.967137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:51:39.967328 (+ 191us) server_negotiation.cc:407] Connection header received
0929 10:51:39.967866 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:51:39.967867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:51:39.967882 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:51:39.967955 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:51:39.968520 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:51:39.969112 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:51:39.970117 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:51:39.970332 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:51:39.970418 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:51:39.973045 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:51:39.973047 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:51:39.973047 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:51:39.973054 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:51:39.974528 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:51:39.975046 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:51:39.975047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:51:39.975048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:51:39.975137 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:51:39.975454 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:51:39.975455 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:51:39.975455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:51:39.975603 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:51:39.975682 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 10:51:39.976050 (+ 368us) server_negotiation.cc:299] Negotiation successful
0929 10:51:39.976145 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":96,"threads_started":1}
I20260929 10:51:39.979568 32316 heartbeater.cc:347] Connected to a master server at 127.20.80.254:32901
I20260929 10:51:39.979697 32316 heartbeater.cc:464] Registering TS with master...
I20260929 10:51:39.980050 32316 heartbeater.cc:511] Master 127.20.80.254:32901 requested a full tablet report, sending...
I20260929 10:51:39.980613 32102 ts_manager.cc:194] Registered new tserver with Master: 629e6ed702cd4fe7a7d347b971737719 (127.20.80.193:38781)
I20260929 10:51:39.981609 32102 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.193@KRBTEST.COM'} at 127.20.80.193:43051
WARNING: no policy specified for kudu/127.20.80.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
I20260929 10:51:40.008517 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.80.212:41997/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.194:0
--local_ip_for_outbound_sockets=127.20.80.194
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:32901
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:33851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:51:40.125761 32322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:51:40.126047 32322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:51:40.126088 32322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:51:40.127905 32322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:51:40.127997 32322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:51:40.128057 32322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.194
I20260929 10:51:40.130236 32322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:33851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.20.80.212:41997/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:32901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.194
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:51:40.130486 32322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:51:40.130766 32322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:51:40.137219 32329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:51:40.137521 32330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
Sep 29 10:51:40 dist-test-slave-vqch krb5kdc[31621](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790679100, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260929 10:51:40.137214 32332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:51:40.138216 32322 server_base.cc:1034] running on GCE node
I20260929 10:51:40.138444 32322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:51:40.138736 32322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:51:40.139930 32322 hybrid_clock.cc:648] HybridClock initialized: now 1790679100139903 us; error 41 us; skew 500 ppm
Sep 29 10:51:40 dist-test-slave-vqch krb5kdc[31621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679100, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:51:40.143285 32322 init.cc:377] Logged in from keytab as kudu/127.20.80.194@KRBTEST.COM (short username kudu)
I20260929 10:51:40.143906 32322 webserver.cc:460] Webserver started at http://127.20.80.194:41665/ using document root <none> and password file <none>
I20260929 10:51:40.144229 32322 fs_manager.cc:359] Metadata directory not provided
I20260929 10:51:40.144302 32322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:51:40.144502 32322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 10:51:40 dist-test-slave-vqch krb5kdc[31621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679100, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for HTTP/127.20.80.212@KRBTEST.COM
I20260929 10:51:40.158463 32322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-1/data/instance:
uuid: "73871865232c4b6197ef74e04bb0eabe"
format_stamp: "Formatted at 2026-09-29 10:51:40 on dist-test-slave-vqch"
server_key: "28d744edfb5fda37ce8bab35fabd4168"
server_key_iv: "f068ce7a3371b23270e5aa3cc4c3bc8d"
server_key_version: "kuduclusterkey@0"
I20260929 10:51:40.158907 32322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance:
uuid: "73871865232c4b6197ef74e04bb0eabe"
format_stamp: "Formatted at 2026-09-29 10:51:40 on dist-test-slave-vqch"
server_key: "28d744edfb5fda37ce8bab35fabd4168"
server_key_iv: "f068ce7a3371b23270e5aa3cc4c3bc8d"
server_key_version: "kuduclusterkey@0"
I20260929 10:51:40.176781 32322 fs_manager.cc:691] Time spent creating directory manager: real 0.018s user 0.002s sys 0.000s
I20260929 10:51:40.191761 32339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:51:40.191957 32322 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:51:40.192042 32322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-1/wal
uuid: "73871865232c4b6197ef74e04bb0eabe"
format_stamp: "Formatted at 2026-09-29 10:51:40 on dist-test-slave-vqch"
server_key: "28d744edfb5fda37ce8bab35fabd4168"
server_key_iv: "f068ce7a3371b23270e5aa3cc4c3bc8d"
server_key_version: "kuduclusterkey@0"
I20260929 10:51:40.192143 32322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:51:40.215919 32322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:51:40.218379 32322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:51:40.218525 32322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:51:40.218724 32322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:51:40.219089 32322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:51:40.219127 32322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:51:40.219157 32322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:51:40.219174 32322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:51:40.225610 32322 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.194:33481
I20260929 10:51:40.226130 32322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
I20260929 10:51:40.227567 32453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.194:33481 every 8 connection(s)
I20260929 10:51:40.235318 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 32322
I20260929 10:51:40.235441 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance
I20260929 10:51:40.235636 20803 external_mini_cluster.cc:1428] Setting key 02fd6ec7d175f01de4a1811fd0976b42
Sep 29 10:51:40 dist-test-slave-vqch krb5kdc[31621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679100, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:51:40.241020 32319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:51:40.229456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:35113 (local address 127.20.80.254:32901)
0929 10:51:40.229509 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0929 10:51:40.229511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:51:40.230674 (+ 1163us) server_negotiation.cc:407] Connection header received
0929 10:51:40.231402 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:51:40.231404 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:51:40.231419 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:51:40.231451 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:51:40.232305 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:51:40.233175 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:51:40.234666 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:51:40.234994 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:51:40.235039 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:51:40.237658 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:51:40.237660 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:51:40.237660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:51:40.237667 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:51:40.239443 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:51:40.239931 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:51:40.239933 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:51:40.239933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:51:40.239992 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:51:40.240263 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:51:40.240264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:51:40.240264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:51:40.240394 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:51:40.240440 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0929 10:51:40.240807 (+ 367us) server_negotiation.cc:299] Negotiation successful
0929 10:51:40.240906 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 10:51:40.241770 32456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:51:40.229164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:32901 (local address 127.20.80.194:35113)
0929 10:51:40.230538 (+ 1374us) negotiation.cc:107] Waiting for socket to connect
0929 10:51:40.230585 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 10:51:40.231273 (+ 688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:51:40.231638 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:51:40.231643 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:51:40.231896 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:51:40.232190 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:51:40.232198 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:51:40.233287 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:51:40.233288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:51:40.234564 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:51:40.234567 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:51:40.235132 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:51:40.235132 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:51:40.235271 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:51:40.235884 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:51:40.235894 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:51:40.237555 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:51:40.239541 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:51:40.239547 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:51:40.239550 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:51:40.239829 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:51:40.240063 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:51:40.240064 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:51:40.240064 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:51:40.240189 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:51:40.240488 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:51:40.240494 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:51:40.240653 (+ 159us) client_negotiation.cc:769] Sending connection context
0929 10:51:40.241370 (+ 717us) client_negotiation.cc:240] Negotiation successful
0929 10:51:40.241534 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1331,"thread_start_us":87,"threads_started":1}
I20260929 10:51:40.242449 32454 heartbeater.cc:347] Connected to a master server at 127.20.80.254:32901
I20260929 10:51:40.242542 32454 heartbeater.cc:464] Registering TS with master...
I20260929 10:51:40.242722 32454 heartbeater.cc:511] Master 127.20.80.254:32901 requested a full tablet report, sending...
I20260929 10:51:40.243149 32102 ts_manager.cc:194] Registered new tserver with Master: 73871865232c4b6197ef74e04bb0eabe (127.20.80.194:33481)
I20260929 10:51:40.243633 32102 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.194@KRBTEST.COM'} at 127.20.80.194:35113
WARNING: no policy specified for kudu/127.20.80.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
I20260929 10:51:40.276947 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.80.212:41997/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.195:0
--local_ip_for_outbound_sockets=127.20.80.195
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:32901
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:33851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:51:40.390942 32459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:51:40.391333 32459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:51:40.391433 32459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:51:40.393323 32459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:51:40.393437 32459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:51:40.393525 32459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.195
I20260929 10:51:40.395897 32459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:33851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.20.80.212:41997/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:32901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.195
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:51:40.396384 32459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:51:40.396714 32459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:51:40.403366 32467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:51:40.404929 32464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:51:40.405009 32465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:51:40.405279 32459 server_base.cc:1034] running on GCE node
I20260929 10:51:40.405504 32459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:51:40.405789 32459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:51:40.409164 32459 hybrid_clock.cc:648] HybridClock initialized: now 1790679100409125 us; error 50 us; skew 500 ppm
Sep 29 10:51:40 dist-test-slave-vqch krb5kdc[31621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679100, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:51:40.412493 32459 init.cc:377] Logged in from keytab as kudu/127.20.80.195@KRBTEST.COM (short username kudu)
I20260929 10:51:40.413146 32459 webserver.cc:460] Webserver started at http://127.20.80.195:42353/ using document root <none> and password file <none>
I20260929 10:51:40.413410 32459 fs_manager.cc:359] Metadata directory not provided
I20260929 10:51:40.413481 32459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:51:40.413661 32459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 10:51:40 dist-test-slave-vqch krb5kdc[31621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679100, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for HTTP/127.20.80.212@KRBTEST.COM
I20260929 10:51:40.427134 32459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-2/data/instance:
uuid: "4acb50649b7749a2838897c2983577c2"
format_stamp: "Formatted at 2026-09-29 10:51:40 on dist-test-slave-vqch"
server_key: "a7b77140a9da66cd71e9f00658dda48b"
server_key_iv: "92f2fa92f66eeabcc30dba6ae787eab7"
server_key_version: "kuduclusterkey@0"
I20260929 10:51:40.427556 32459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance:
uuid: "4acb50649b7749a2838897c2983577c2"
format_stamp: "Formatted at 2026-09-29 10:51:40 on dist-test-slave-vqch"
server_key: "a7b77140a9da66cd71e9f00658dda48b"
server_key_iv: "92f2fa92f66eeabcc30dba6ae787eab7"
server_key_version: "kuduclusterkey@0"
I20260929 10:51:40.429083 32459 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:51:40.438831 32474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:51:40.439236 32459 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 10:51:40.439338 32459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-2/wal
uuid: "4acb50649b7749a2838897c2983577c2"
format_stamp: "Formatted at 2026-09-29 10:51:40 on dist-test-slave-vqch"
server_key: "a7b77140a9da66cd71e9f00658dda48b"
server_key_iv: "92f2fa92f66eeabcc30dba6ae787eab7"
server_key_version: "kuduclusterkey@0"
I20260929 10:51:40.439430 32459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:51:40.479990 32459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:51:40.482777 32459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:51:40.483044 32459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:51:40.483400 32459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:51:40.483866 32459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:51:40.483968 32459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:51:40.484045 32459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:51:40.484107 32459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:51:40.491590 32459 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.195:40705
I20260929 10:51:40.492251 32459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
I20260929 10:51:40.494195 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 32459
I20260929 10:51:40.494302 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance
I20260929 10:51:40.494494 20803 external_mini_cluster.cc:1428] Setting key 8d9d5b6a83f04ce75bc3da2c72f78ea1
I20260929 10:51:40.495210 32589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.195:40705 every 8 connection(s)
Sep 29 10:51:40 dist-test-slave-vqch krb5kdc[31621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679100, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:51:40.508152 32319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:51:40.496279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:58077 (local address 127.20.80.254:32901)
0929 10:51:40.496322 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 10:51:40.496324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:51:40.497253 (+ 929us) server_negotiation.cc:407] Connection header received
0929 10:51:40.498026 (+ 773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:51:40.498027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:51:40.498040 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:51:40.498125 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:51:40.498920 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:51:40.499726 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:51:40.501361 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:51:40.501660 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:51:40.501709 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:51:40.504534 (+ 2825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:51:40.504536 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:51:40.504536 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:51:40.504544 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:51:40.506439 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:51:40.506961 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:51:40.506963 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:51:40.506963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:51:40.507032 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:51:40.507369 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:51:40.507370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:51:40.507370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:51:40.507521 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:51:40.507601 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 10:51:40.507959 (+ 358us) server_negotiation.cc:299] Negotiation successful
0929 10:51:40.508056 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 10:51:40.508315 32592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:51:40.496002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:32901 (local address 127.20.80.195:58077)
0929 10:51:40.497106 (+ 1104us) negotiation.cc:107] Waiting for socket to connect
0929 10:51:40.497162 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0929 10:51:40.497898 (+ 736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:51:40.498230 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:51:40.498234 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:51:40.498493 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:51:40.498794 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:51:40.498801 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:51:40.499845 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:51:40.499846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:51:40.501248 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:51:40.501251 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:51:40.501810 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:51:40.501811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:51:40.501972 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:51:40.502635 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:51:40.502646 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:51:40.504413 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:51:40.506542 (+ 2129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:51:40.506550 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:51:40.506552 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:51:40.506870 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:51:40.507127 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:51:40.507128 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:51:40.507129 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:51:40.507277 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:51:40.507610 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:51:40.507616 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:51:40.507787 (+ 171us) client_negotiation.cc:769] Sending connection context
0929 10:51:40.507911 (+ 124us) client_negotiation.cc:240] Negotiation successful
0929 10:51:40.508079 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1054,"thread_start_us":92,"threads_started":1}
I20260929 10:51:40.508919 32590 heartbeater.cc:347] Connected to a master server at 127.20.80.254:32901
I20260929 10:51:40.509047 32590 heartbeater.cc:464] Registering TS with master...
I20260929 10:51:40.509243 32590 heartbeater.cc:511] Master 127.20.80.254:32901 requested a full tablet report, sending...
I20260929 10:51:40.509658 32102 ts_manager.cc:194] Registered new tserver with Master: 4acb50649b7749a2838897c2983577c2 (127.20.80.195:40705)
I20260929 10:51:40.510167 32102 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.195@KRBTEST.COM'} at 127.20.80.195:58077
I20260929 10:51:40.517323 20803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:51:40.527299 32319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:51:40.519258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54062 (local address 127.20.80.254:32901)
0929 10:51:40.519312 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0929 10:51:40.519314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:51:40.519524 (+ 210us) server_negotiation.cc:407] Connection header received
0929 10:51:40.519573 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:51:40.519574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:51:40.519586 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:51:40.519642 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:51:40.520343 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:51:40.521141 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:51:40.522239 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:51:40.522527 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:51:40.522599 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:51:40.524219 (+ 1620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:51:40.524221 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:51:40.524221 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:51:40.524229 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:51:40.526016 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:51:40.526429 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:51:40.526430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:51:40.526431 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:51:40.526487 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:51:40.526690 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:51:40.526691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:51:40.526692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:51:40.526828 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:51:40.526903 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 10:51:40.527126 (+ 223us) server_negotiation.cc:299] Negotiation successful
0929 10:51:40.527203 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260929 10:51:40.529731 32102 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54062:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:51:40.532130 32102 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:51:40.540663 32606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:51:40.535012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:40705 (local address 127.0.0.1:49838)
0929 10:51:40.536193 (+ 1181us) negotiation.cc:107] Waiting for socket to connect
0929 10:51:40.536234 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 10:51:40.536314 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:51:40.536616 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:51:40.536620 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:51:40.536622 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:51:40.536693 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:51:40.536695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:51:40.538009 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:51:40.538010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:51:40.539298 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:51:40.539301 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:51:40.540368 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:51:40.540369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:51:40.540450 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:51:40.540464 (+ 14us) client_negotiation.cc:769] Sending connection context
0929 10:51:40.540534 (+ 70us) client_negotiation.cc:240] Negotiation successful
0929 10:51:40.540580 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1151,"thread_start_us":72,"threads_started":1}
I20260929 10:51:40.541144 32605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:51:40.535445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49838 (local address 127.20.80.195:40705)
0929 10:51:40.535694 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0929 10:51:40.535700 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:51:40.536320 (+ 620us) server_negotiation.cc:407] Connection header received
0929 10:51:40.536409 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:51:40.536411 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:51:40.536430 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:51:40.536522 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:51:40.536774 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:51:40.537908 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:51:40.539424 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:51:40.540182 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:51:40.540255 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:51:40.540331 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 10:51:40.541017 (+ 686us) server_negotiation.cc:299] Negotiation successful
0929 10:51:40.541059 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":83,"threads_started":1}
I20260929 10:51:40.541420 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:51:40.535555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:33481 (local address 127.0.0.1:34646)
0929 10:51:40.536564 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0929 10:51:40.536571 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0929 10:51:40.536610 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:51:40.536882 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:51:40.536883 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:51:40.536885 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:51:40.536929 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:51:40.536930 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:51:40.538567 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:51:40.538568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:51:40.539743 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:51:40.539746 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:51:40.541174 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:51:40.541175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:51:40.541236 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:51:40.541241 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 10:51:40.541299 (+ 58us) client_negotiation.cc:240] Negotiation successful
0929 10:51:40.541336 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":996,"thread_start_us":76,"threads_started":1}
I20260929 10:51:40.541895 32607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:51:40.535701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34646 (local address 127.20.80.194:33481)
0929 10:51:40.535900 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0929 10:51:40.535905 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:51:40.536664 (+ 759us) server_negotiation.cc:407] Connection header received
0929 10:51:40.536711 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:51:40.536713 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:51:40.536730 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:51:40.536803 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:51:40.537055 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:51:40.538465 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:51:40.539862 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:51:40.540937 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:51:40.541067 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:51:40.541146 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 10:51:40.541771 (+ 625us) server_negotiation.cc:299] Negotiation successful
0929 10:51:40.541813 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":104,"threads_started":1}
I20260929 10:51:40.542136 32522 tablet_service.cc:1467] Processing CreateTablet for tablet 917e1a9d8081417188bc5325364ca0a6 (DEFAULT_TABLE table=test-table [id=b7de7b5ef878472e8316b3cd41186a6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:51:40.542501 32522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 917e1a9d8081417188bc5325364ca0a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:51:40.542645 32386 tablet_service.cc:1467] Processing CreateTablet for tablet 917e1a9d8081417188bc5325364ca0a6 (DEFAULT_TABLE table=test-table [id=b7de7b5ef878472e8316b3cd41186a6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:51:40.543018 32386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 917e1a9d8081417188bc5325364ca0a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:51:40.546419 32608 tablet_bootstrap.cc:492] T 917e1a9d8081417188bc5325364ca0a6 P 4acb50649b7749a2838897c2983577c2: Bootstrap starting.
I20260929 10:51:40.547447 32608 tablet_bootstrap.cc:654] T 917e1a9d8081417188bc5325364ca0a6 P 4acb50649b7749a2838897c2983577c2: Neither blocks nor log segments found. Creating new log.
I20260929 10:51:40.547821 32608 log.cc:824] T 917e1a9d8081417188bc5325364ca0a6 P 4acb50649b7749a2838897c2983577c2: Log is configured to *not* fsync() on all Append() calls
I20260929 10:51:40.548635 32608 tablet_bootstrap.cc:492] T 917e1a9d8081417188bc5325364ca0a6 P 4acb50649b7749a2838897c2983577c2: No bootstrap required, opened a new log
I20260929 10:51:40.548698 32608 ts_tablet_manager.cc:1397] T 917e1a9d8081417188bc5325364ca0a6 P 4acb50649b7749a2838897c2983577c2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:51:40.550424 32603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:51:40.535234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:38781 (local address 127.0.0.1:43190)
0929 10:51:40.536445 (+ 1211us) negotiation.cc:107] Waiting for socket to connect
0929 10:51:40.536454 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:51:40.536492 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:51:40.546607 (+ 10115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:51:40.546608 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:51:40.546610 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:51:40.546646 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:51:40.546648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:51:40.548126 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:51:40.548127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:51:40.549265 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:51:40.549268 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:51:40.550212 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:51:40.550213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:51:40.550272 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:51:40.550276 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:51:40.550305 (+ 29us) client_negotiation.cc:240] Negotiation successful
0929 10:51:40.550341 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1196,"thread_start_us":62,"threads_started":1}
I20260929 10:51:40.550740 32604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:51:40.535337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43190 (local address 127.20.80.193:38781)
0929 10:51:40.545104 (+ 9767us) server_negotiation.cc:206] Beginning negotiation
0929 10:51:40.545110 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:51:40.545122 (+ 12us) server_negotiation.cc:407] Connection header received
0929 10:51:40.545157 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:51:40.545159 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:51:40.545176 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:51:40.545260 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:51:40.546742 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:51:40.547820 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:51:40.549369 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:51:40.550108 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:51:40.550527 (+ 419us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:51:40.550600 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 10:51:40.550628 (+ 28us) server_negotiation.cc:299] Negotiation successful
0929 10:51:40.550671 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9726,"thread_start_us":66,"threads_started":1}
I20260929 10:51:40.550753 32608 raft_consensus.cc:348] T 917e1a9d8081417188bc5325364ca0a6 P 4acb50649b7749a2838897c2983577c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acb50649b7749a2838897c2983577c2" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40705 } } peers { permanent_uuid: "73871865232c4b6197ef74e04bb0eabe" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33481 } } peers { permanent_uuid: "629e6ed702cd4fe7a7d347b971737719" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 38781 } }
I20260929 10:51:40.550936 32608 raft_consensus.cc:374] T 917e1a9d8081417188bc5325364ca0a6 P 4acb50649b7749a2838897c2983577c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:51:40.550945 32609 tablet_bootstrap.cc:492] T 917e1a9d8081417188bc5325364ca0a6 P 73871865232c4b6197ef74e04bb0eabe: Bootstrap starting.
I20260929 10:51:40.550974 32608 raft_consensus.cc:729] T 917e1a9d8081417188bc5325364ca0a6 P 4acb50649b7749a2838897c2983577c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4acb50649b7749a2838897c2983577c2, State: Initialized, Role: FOLLOWER
I20260929 10:51:40.551085 32608 consensus_queue.cc:260] T 917e1a9d8081417188bc5325364ca0a6 P 4acb50649b7749a2838897c2983577c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acb50649b7749a2838897c2983577c2" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40705 } } peers { permanent_uuid: "73871865232c4b6197ef74e04bb0eabe" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33481 } } peers { permanent_uuid: "629e6ed702cd4fe7a7d347b971737719" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 38781 } }
I20260929 10:51:40.551340 32608 ts_tablet_manager.cc:1428] T 917e1a9d8081417188bc5325364ca0a6 P 4acb50649b7749a2838897c2983577c2: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260929 10:51:40.551743 32590 heartbeater.cc:503] Master 127.20.80.254:32901 was elected leader, sending a full tablet report...
I20260929 10:51:40.552104 32234 tablet_service.cc:1467] Processing CreateTablet for tablet 917e1a9d8081417188bc5325364ca0a6 (DEFAULT_TABLE table=test-table [id=b7de7b5ef878472e8316b3cd41186a6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:51:40.552271 32609 tablet_bootstrap.cc:654] T 917e1a9d8081417188bc5325364ca0a6 P 73871865232c4b6197ef74e04bb0eabe: Neither blocks nor log segments found. Creating new log.
I20260929 10:51:40.552461 32234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 917e1a9d8081417188bc5325364ca0a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:51:40.552618 32609 log.cc:824] T 917e1a9d8081417188bc5325364ca0a6 P 73871865232c4b6197ef74e04bb0eabe: Log is configured to *not* fsync() on all Append() calls
I20260929 10:51:40.555418 32613 tablet_bootstrap.cc:492] T 917e1a9d8081417188bc5325364ca0a6 P 629e6ed702cd4fe7a7d347b971737719: Bootstrap starting.
I20260929 10:51:40.556143 32613 tablet_bootstrap.cc:654] T 917e1a9d8081417188bc5325364ca0a6 P 629e6ed702cd4fe7a7d347b971737719: Neither blocks nor log segments found. Creating new log.
I20260929 10:51:40.556456 32613 log.cc:824] T 917e1a9d8081417188bc5325364ca0a6 P 629e6ed702cd4fe7a7d347b971737719: Log is configured to *not* fsync() on all Append() calls
I20260929 10:51:40.557317 32613 tablet_bootstrap.cc:492] T 917e1a9d8081417188bc5325364ca0a6 P 629e6ed702cd4fe7a7d347b971737719: No bootstrap required, opened a new log
I20260929 10:51:40.557390 32613 ts_tablet_manager.cc:1397] T 917e1a9d8081417188bc5325364ca0a6 P 629e6ed702cd4fe7a7d347b971737719: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:51:40.561908 32613 raft_consensus.cc:348] T 917e1a9d8081417188bc5325364ca0a6 P 629e6ed702cd4fe7a7d347b971737719 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acb50649b7749a2838897c2983577c2" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40705 } } peers { permanent_uuid: "73871865232c4b6197ef74e04bb0eabe" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33481 } } peers { permanent_uuid: "629e6ed702cd4fe7a7d347b971737719" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 38781 } }
I20260929 10:51:40.562114 32613 raft_consensus.cc:374] T 917e1a9d8081417188bc5325364ca0a6 P 629e6ed702cd4fe7a7d347b971737719 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:51:40.562191 32613 raft_consensus.cc:729] T 917e1a9d8081417188bc5325364ca0a6 P 629e6ed702cd4fe7a7d347b971737719 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 629e6ed702cd4fe7a7d347b971737719, State: Initialized, Role: FOLLOWER
I20260929 10:51:40.562343 32613 consensus_queue.cc:260] T 917e1a9d8081417188bc5325364ca0a6 P 629e6ed702cd4fe7a7d347b971737719 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acb50649b7749a2838897c2983577c2" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40705 } } peers { permanent_uuid: "73871865232c4b6197ef74e04bb0eabe" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33481 } } peers { permanent_uuid: "629e6ed702cd4fe7a7d347b971737719" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 38781 } }
I20260929 10:51:40.562693 32613 ts_tablet_manager.cc:1428] T 917e1a9d8081417188bc5325364ca0a6 P 629e6ed702cd4fe7a7d347b971737719: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260929 10:51:40.562721 32316 heartbeater.cc:503] Master 127.20.80.254:32901 was elected leader, sending a full tablet report...
I20260929 10:51:40.565801 32609 tablet_bootstrap.cc:492] T 917e1a9d8081417188bc5325364ca0a6 P 73871865232c4b6197ef74e04bb0eabe: No bootstrap required, opened a new log
I20260929 10:51:40.565883 32609 ts_tablet_manager.cc:1397] T 917e1a9d8081417188bc5325364ca0a6 P 73871865232c4b6197ef74e04bb0eabe: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20260929 10:51:40.569954 32609 raft_consensus.cc:348] T 917e1a9d8081417188bc5325364ca0a6 P 73871865232c4b6197ef74e04bb0eabe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acb50649b7749a2838897c2983577c2" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40705 } } peers { permanent_uuid: "73871865232c4b6197ef74e04bb0eabe" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33481 } } peers { permanent_uuid: "629e6ed702cd4fe7a7d347b971737719" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 38781 } }
I20260929 10:51:40.570173 32609 raft_consensus.cc:374] T 917e1a9d8081417188bc5325364ca0a6 P 73871865232c4b6197ef74e04bb0eabe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:51:40.570212 32609 raft_consensus.cc:729] T 917e1a9d8081417188bc5325364ca0a6 P 73871865232c4b6197ef74e04bb0eabe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73871865232c4b6197ef74e04bb0eabe, State: Initialized, Role: FOLLOWER
I20260929 10:51:40.570348 32609 consensus_queue.cc:260] T 917e1a9d8081417188bc5325364ca0a6 P 73871865232c4b6197ef74e04bb0eabe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acb50649b7749a2838897c2983577c2" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40705 } } peers { permanent_uuid: "73871865232c4b6197ef74e04bb0eabe" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33481 } } peers { permanent_uuid: "629e6ed702cd4fe7a7d347b971737719" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 38781 } }
I20260929 10:51:40.570765 32454 heartbeater.cc:503] Master 127.20.80.254:32901 was elected leader, sending a full tablet report...
I20260929 10:51:40.570720 32609 ts_tablet_manager.cc:1428] T 917e1a9d8081417188bc5325364ca0a6 P 73871865232c4b6197ef74e04bb0eabe: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
Sep 29 10:51:40 dist-test-slave-vqch krb5kdc[31621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679100, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for HTTP/127.20.80.212@KRBTEST.COM
I20260929 10:51:40.692886 32615 raft_consensus.cc:482] T 917e1a9d8081417188bc5325364ca0a6 P 629e6ed702cd4fe7a7d347b971737719 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:51:40.693078 32615 raft_consensus.cc:504] T 917e1a9d8081417188bc5325364ca0a6 P 629e6ed702cd4fe7a7d347b971737719 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acb50649b7749a2838897c2983577c2" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40705 } } peers { permanent_uuid: "73871865232c4b6197ef74e04bb0eabe" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33481 } } peers { permanent_uuid: "629e6ed702cd4fe7a7d347b971737719" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 38781 } }
I20260929 10:51:40.693655 32615 leader_election.cc:288] T 917e1a9d8081417188bc5325364ca0a6 P 629e6ed702cd4fe7a7d347b971737719 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4acb50649b7749a2838897c2983577c2 (127.20.80.195:40705), 73871865232c4b6197ef74e04bb0eabe (127.20.80.194:33481)
I20260929 10:51:40.698345 32605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:51:40.694005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:58351 (local address 127.20.80.195:40705)
0929 10:51:40.694052 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 10:51:40.694055 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:51:40.694607 (+ 552us) server_negotiation.cc:407] Connection header received
0929 10:51:40.694709 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:51:40.694710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:51:40.694722 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:51:40.694753 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:51:40.695011 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:51:40.695754 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:51:40.697363 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:51:40.697930 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:51:40.697984 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:51:40.697996 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0929 10:51:40.698212 (+ 216us) server_negotiation.cc:299] Negotiation successful
0929 10:51:40.698271 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 10:51:40.698367 32607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:51:40.694073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:60599 (local address 127.20.80.194:33481)
0929 10:51:40.694121 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 10:51:40.694123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:51:40.694551 (+ 428us) server_negotiation.cc:407] Connection header received
0929 10:51:40.694671 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:51:40.694672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:51:40.694689 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:51:40.694720 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:51:40.695024 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:51:40.695786 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:51:40.697428 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:51:40.697991 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:51:40.698037 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:51:40.698045 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 10:51:40.698262 (+ 217us) server_negotiation.cc:299] Negotiation successful
0929 10:51:40.698300 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 10:51:40.698599 32617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:51:40.693860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:33481 (local address 127.20.80.193:60599)
0929 10:51:40.694464 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0929 10:51:40.694476 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:51:40.694586 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:51:40.694829 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:51:40.694829 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:51:40.694831 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:51:40.694947 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:51:40.694949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:51:40.695951 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:51:40.695952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:51:40.697335 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:51:40.697338 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:51:40.698102 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:51:40.698103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:51:40.698185 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:51:40.698188 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:51:40.698471 (+ 283us) client_negotiation.cc:240] Negotiation successful
0929 10:51:40.698511 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"mutex_wait_us":14,"thread_start_us":442,"threads_started":1}
I20260929 10:51:40.698897 32618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:51:40.693893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:40705 (local address 127.20.80.193:58351)
0929 10:51:40.694538 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0929 10:51:40.694548 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:51:40.694634 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:51:40.694827 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:51:40.694828 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:51:40.694830 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:51:40.694924 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:51:40.694926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:51:40.695887 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:51:40.695888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:51:40.697251 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:51:40.697265 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:51:40.698060 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:51:40.698061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:51:40.698130 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:51:40.698133 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:51:40.698766 (+ 633us) client_negotiation.cc:240] Negotiation successful
0929 10:51:40.698814 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"mutex_wait_us":46,"thread_start_us":464,"threads_started":1}
I20260929 10:51:40.698994 32391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "917e1a9d8081417188bc5325364ca0a6" candidate_uuid: "629e6ed702cd4fe7a7d347b971737719" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73871865232c4b6197ef74e04bb0eabe" is_pre_election: true
I20260929 10:51:40.699160 32391 raft_consensus.cc:2454] T 917e1a9d8081417188bc5325364ca0a6 P 73871865232c4b6197ef74e04bb0eabe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 629e6ed702cd4fe7a7d347b971737719 in term 0.
I20260929 10:51:40.699393 32202 leader_election.cc:302] T 917e1a9d8081417188bc5325364ca0a6 P 629e6ed702cd4fe7a7d347b971737719 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 629e6ed702cd4fe7a7d347b971737719, 73871865232c4b6197ef74e04bb0eabe; no voters:
I20260929 10:51:40.699371 32542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "917e1a9d8081417188bc5325364ca0a6" candidate_uuid: "629e6ed702cd4fe7a7d347b971737719" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4acb50649b7749a2838897c2983577c2" is_pre_election: true
I20260929 10:51:40.699505 32542 raft_consensus.cc:2454] T 917e1a9d8081417188bc5325364ca0a6 P 4acb50649b7749a2838897c2983577c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 629e6ed702cd4fe7a7d347b971737719 in term 0.
I20260929 10:51:40.699543 32615 raft_consensus.cc:2781] T 917e1a9d8081417188bc5325364ca0a6 P 629e6ed702cd4fe7a7d347b971737719 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:51:40.699581 32615 raft_consensus.cc:482] T 917e1a9d8081417188bc5325364ca0a6 P 629e6ed702cd4fe7a7d347b971737719 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:51:40.699602 32615 raft_consensus.cc:3037] T 917e1a9d8081417188bc5325364ca0a6 P 629e6ed702cd4fe7a7d347b971737719 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:51:40.700490 32615 raft_consensus.cc:504] T 917e1a9d8081417188bc5325364ca0a6 P 629e6ed702cd4fe7a7d347b971737719 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acb50649b7749a2838897c2983577c2" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40705 } } peers { permanent_uuid: "73871865232c4b6197ef74e04bb0eabe" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33481 } } peers { permanent_uuid: "629e6ed702cd4fe7a7d347b971737719" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 38781 } }
I20260929 10:51:40.700652 32615 leader_election.cc:288] T 917e1a9d8081417188bc5325364ca0a6 P 629e6ed702cd4fe7a7d347b971737719 [CANDIDATE]: Term 1 election: Requested vote from peers 4acb50649b7749a2838897c2983577c2 (127.20.80.195:40705), 73871865232c4b6197ef74e04bb0eabe (127.20.80.194:33481)
I20260929 10:51:40.700846 32391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "917e1a9d8081417188bc5325364ca0a6" candidate_uuid: "629e6ed702cd4fe7a7d347b971737719" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73871865232c4b6197ef74e04bb0eabe"
I20260929 10:51:40.700912 32391 raft_consensus.cc:3037] T 917e1a9d8081417188bc5325364ca0a6 P 73871865232c4b6197ef74e04bb0eabe [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:51:40.701187 32542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "917e1a9d8081417188bc5325364ca0a6" candidate_uuid: "629e6ed702cd4fe7a7d347b971737719" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4acb50649b7749a2838897c2983577c2"
I20260929 10:51:40.701272 32542 raft_consensus.cc:3037] T 917e1a9d8081417188bc5325364ca0a6 P 4acb50649b7749a2838897c2983577c2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:51:40.701751 32391 raft_consensus.cc:2454] T 917e1a9d8081417188bc5325364ca0a6 P 73871865232c4b6197ef74e04bb0eabe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 629e6ed702cd4fe7a7d347b971737719 in term 1.
I20260929 10:51:40.701911 32202 leader_election.cc:302] T 917e1a9d8081417188bc5325364ca0a6 P 629e6ed702cd4fe7a7d347b971737719 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 629e6ed702cd4fe7a7d347b971737719, 73871865232c4b6197ef74e04bb0eabe; no voters:
I20260929 10:51:40.701994 32615 raft_consensus.cc:2781] T 917e1a9d8081417188bc5325364ca0a6 P 629e6ed702cd4fe7a7d347b971737719 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:51:40.702126 32615 raft_consensus.cc:686] T 917e1a9d8081417188bc5325364ca0a6 P 629e6ed702cd4fe7a7d347b971737719 [term 1 LEADER]: Becoming Leader. State: Replica: 629e6ed702cd4fe7a7d347b971737719, State: Running, Role: LEADER
I20260929 10:51:40.702175 32542 raft_consensus.cc:2454] T 917e1a9d8081417188bc5325364ca0a6 P 4acb50649b7749a2838897c2983577c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 629e6ed702cd4fe7a7d347b971737719 in term 1.
I20260929 10:51:40.702210 32615 consensus_queue.cc:237] T 917e1a9d8081417188bc5325364ca0a6 P 629e6ed702cd4fe7a7d347b971737719 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acb50649b7749a2838897c2983577c2" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40705 } } peers { permanent_uuid: "73871865232c4b6197ef74e04bb0eabe" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33481 } } peers { permanent_uuid: "629e6ed702cd4fe7a7d347b971737719" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 38781 } }
I20260929 10:51:40.702993 32102 catalog_manager.cc:5495] T 917e1a9d8081417188bc5325364ca0a6 P 629e6ed702cd4fe7a7d347b971737719 reported cstate change: term changed from 0 to 1, leader changed from <none> to 629e6ed702cd4fe7a7d347b971737719 (127.20.80.193). New cstate: current_term: 1 leader_uuid: "629e6ed702cd4fe7a7d347b971737719" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acb50649b7749a2838897c2983577c2" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73871865232c4b6197ef74e04bb0eabe" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "629e6ed702cd4fe7a7d347b971737719" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 38781 } health_report { overall_health: HEALTHY } } }
W20260929 10:51:40.717505 32317 tablet.cc:2369] T 917e1a9d8081417188bc5325364ca0a6 P 629e6ed702cd4fe7a7d347b971737719: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:51:40.720924 32604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:51:40.718381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43196 (local address 127.20.80.193:38781)
0929 10:51:40.718424 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 10:51:40.718426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:51:40.718436 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:51:40.718519 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:51:40.718520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:51:40.718540 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:51:40.718608 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:51:40.718776 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:51:40.719279 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:51:40.720087 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:51:40.720299 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:51:40.720391 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:51:40.720569 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:51:40.720675 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:51:40.720769 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 10:51:40.720794 (+ 25us) server_negotiation.cc:299] Negotiation successful
0929 10:51:40.720814 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 10:51:40.723027 32542 raft_consensus.cc:1261] T 917e1a9d8081417188bc5325364ca0a6 P 4acb50649b7749a2838897c2983577c2 [term 1 FOLLOWER]: Refusing update from remote peer 629e6ed702cd4fe7a7d347b971737719: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:51:40.723419 32391 raft_consensus.cc:1261] T 917e1a9d8081417188bc5325364ca0a6 P 73871865232c4b6197ef74e04bb0eabe [term 1 FOLLOWER]: Refusing update from remote peer 629e6ed702cd4fe7a7d347b971737719: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:51:40.723414 32619 consensus_queue.cc:1035] T 917e1a9d8081417188bc5325364ca0a6 P 629e6ed702cd4fe7a7d347b971737719 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4acb50649b7749a2838897c2983577c2" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 40705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:51:40.725034 32619 consensus_queue.cc:1035] T 917e1a9d8081417188bc5325364ca0a6 P 629e6ed702cd4fe7a7d347b971737719 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73871865232c4b6197ef74e04bb0eabe" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 33481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 10:51:40.727402 32455 tablet.cc:2369] T 917e1a9d8081417188bc5325364ca0a6 P 73871865232c4b6197ef74e04bb0eabe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:51:40.728392 32625 mvcc.cc:205] Tried to move back new op lower bound from 7334621596558757888 to 7334621596476780544. Current Snapshot: MvccSnapshot[applied={T|T < 7334621596558757888}]
I20260929 10:51:40.729048 32623 mvcc.cc:205] Tried to move back new op lower bound from 7334621596558757888 to 7334621596476780544. Current Snapshot: MvccSnapshot[applied={T|T < 7334621596558757888}]
I20260929 10:51:40.730806 32102 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:51:40.730897 32102 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:51:40.731639 32102 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 53b931aec02a4231b3c8722955953981: Sending DeleteTablet for 3 replicas of tablet 917e1a9d8081417188bc5325364ca0a6
I20260929 10:51:40.732038 32522 tablet_service.cc:1514] Processing DeleteTablet for tablet 917e1a9d8081417188bc5325364ca0a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:51:40 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:49838
I20260929 10:51:40.732036 32386 tablet_service.cc:1514] Processing DeleteTablet for tablet 917e1a9d8081417188bc5325364ca0a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:51:40 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:34646
I20260929 10:51:40.732081 32234 tablet_service.cc:1514] Processing DeleteTablet for tablet 917e1a9d8081417188bc5325364ca0a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:51:40 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:43190
I20260929 10:51:40.732368 32631 tablet_replica.cc:330] T 917e1a9d8081417188bc5325364ca0a6 P 4acb50649b7749a2838897c2983577c2: stopping tablet replica
I20260929 10:51:40.732431 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 32184
I20260929 10:51:40.732528 32631 raft_consensus.cc:2229] T 917e1a9d8081417188bc5325364ca0a6 P 4acb50649b7749a2838897c2983577c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:51:40.732645 32631 raft_consensus.cc:2258] T 917e1a9d8081417188bc5325364ca0a6 P 4acb50649b7749a2838897c2983577c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:51:40.734328 32631 ts_tablet_manager.cc:1905] T 917e1a9d8081417188bc5325364ca0a6 P 4acb50649b7749a2838897c2983577c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:51:40.735944 32631 ts_tablet_manager.cc:1918] T 917e1a9d8081417188bc5325364ca0a6 P 4acb50649b7749a2838897c2983577c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:51:40.736033 32631 log.cc:1196] T 917e1a9d8081417188bc5325364ca0a6 P 4acb50649b7749a2838897c2983577c2: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790678996126930-20803-0/minicluster-data/ts-2/wal/wals/917e1a9d8081417188bc5325364ca0a6
I20260929 10:51:40.736550 32631 ts_tablet_manager.cc:1939] T 917e1a9d8081417188bc5325364ca0a6 P 4acb50649b7749a2838897c2983577c2: Deleting consensus metadata
I20260929 10:51:40.736961 32086 catalog_manager.cc:4841] TS 4acb50649b7749a2838897c2983577c2 (127.20.80.195:40705): tablet 917e1a9d8081417188bc5325364ca0a6 (table test-table [id=b7de7b5ef878472e8316b3cd41186a6c]) successfully deleted
W20260929 10:51:40.740193 32086 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.80.193:38781 (error 108)
I20260929 10:51:40.740238 32632 tablet_replica.cc:330] T 917e1a9d8081417188bc5325364ca0a6 P 73871865232c4b6197ef74e04bb0eabe: stopping tablet replica
I20260929 10:51:40.740345 32632 raft_consensus.cc:2229] T 917e1a9d8081417188bc5325364ca0a6 P 73871865232c4b6197ef74e04bb0eabe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:51:40.740475 32632 raft_consensus.cc:2258] T 917e1a9d8081417188bc5325364ca0a6 P 73871865232c4b6197ef74e04bb0eabe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:51:40.740686 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 32322
I20260929 10:51:40.740876 32603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:51:40.740746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:38781 (local address 127.0.0.1:43204)
0929 10:51:40.740793 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0929 10:51:40.740825 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.193:38781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260929 10:51:40.741472 32086 catalog_manager.cc:4584] TS 629e6ed702cd4fe7a7d347b971737719 (127.20.80.193:38781): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 917e1a9d8081417188bc5325364ca0a6: Network error: Client connection negotiation failed: client connection to 127.20.80.193:38781: connect: Connection refused (error 111)
I20260929 10:51:40.747135 32603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:51:40.746998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:33481 (local address 127.0.0.1:34648)
0929 10:51:40.747072 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0929 10:51:40.747100 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.194:33481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260929 10:51:40.747116 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 32459
I20260929 10:51:40.753278 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 32070
I20260929 10:51:40.758764 20803 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260929 10:51:41.093341 20803 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260929 10:51:41.093444 20803 mini_ranger.cc:66] Stopping Ranger...
I20260929 10:51:41.437966 20803 mini_ranger.cc:68] Stopped Ranger
2026-09-29 10:51:41.438 UTC [31651] LOG: received smart shutdown request
2026-09-29 10:51:41.442 UTC [31651] LOG: background worker "logical replication launcher" (PID 31660) exited with exit code 1
2026-09-29 10:51:41.442 UTC [31655] LOG: shutting down
2026-09-29 10:51:41.627 UTC [31651] LOG: database system is shut down
2026-09-29T10:51:41Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47597 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:51:41 dist-test-slave-vqch krb5kdc[32643](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:51:41 dist-test-slave-vqch krb5kdc[32643](info): set up 2 sockets
Sep 29 10:51:41 dist-test-slave-vqch krb5kdc[32643](info): commencing operation
krb5kdc: starting...
W20260929 10:51:42.776647 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:51:42 dist-test-slave-vqch krb5kdc[32643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:51:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:51:42Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.20.80.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.20.80.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.20.80.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/rangeradmin_127.20.80.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.20.80.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/rangeradmin_127.20.80.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.20.80.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.20.80.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.20.80.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/rangerlookup_127.20.80.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.20.80.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/rangerlookup_127.20.80.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.80.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.80.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.80.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/HTTP_127.20.80.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.80.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/HTTP_127.20.80.212@KRBTEST.COM.keytab.
I20260929 10:51:42.945989 20803 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskfQc4Oq/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/postgres -l logfile start
2026-09-29 10:51:45.575 UTC [32674] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-29 10:51:45.575 UTC [32674] LOG: listening on IPv4 address "127.20.80.212", port 48009
2026-09-29 10:51:45.579 UTC [32674] LOG: listening on Unix socket "/tmp/.s.PGSQL.48009"
2026-09-29 10:51:45.602 UTC [32677] LOG: database system was shut down at 2026-09-29 10:51:44 UTC
2026-09-29 10:51:45.606 UTC [32674] LOG: database system is ready to accept connections
I20260929 10:51:46.556516 20803 mini_postgres.cc:96] Postgres bound to 48009
2026-09-29 10:51:46.562 UTC [32685] FATAL: database "slave" does not exist
127.20.80.212:48009 - accepting connections
I20260929 10:51:46.563063 20803 mini_ranger.cc:161] Starting Ranger...
I20260929 10:51:46.577785 20803 mini_ranger.cc:84] Created miniranger Postgres user
I20260929 10:51:47.094815 20803 mini_ranger.cc:87] Created ranger Postgres database
I20260929 10:51:47.094880 20803 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ranger-admin
2026-09-29 10:51:47,368 [I] DB FLAVOR :POSTGRES
2026-09-29 10:51:47,368 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 10:51:47,368 [I] Checking connection..
2026-09-29 10:51:47,368 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:48009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-29 10:51:47,634 [I] Checking connection passed.
2026-09-29 10:51:47,634 [I] --------- Verifying version history table ---------
2026-09-29 10:51:47,634 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:48009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 10:51:47,896 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 10:51:47,897 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:48009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 10:51:48,143 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 10:51:48,144 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-29 10:51:48,144 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:48009/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-29 10:51:48.405 UTC [32750] WARNING: there is no transaction in progress
2026-09-29 10:51:48,411 [I] create_dbversion_catalog.sql file imported successfully
2026-09-29 10:51:48,411 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:48009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 10:51:48,646 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-29 10:51:48,646 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 10:51:48,646 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:48009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-29 10:51:48,915 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:48009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-29 10:51:49,173 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:48009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-vqch.c.gcp-upstream.internal','N') ;"
2026-09-29 10:51:49,404 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:48009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 10:51:49,670 [I] Table x_portal_user does not exist in database ranger
2026-09-29 10:51:49,670 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:48009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 10:51:49,921 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-29 10:51:49,921 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:48009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 10:51:50,161 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:48009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 10:51:50,419 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-29 10:51:50,420 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:48009/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-29 10:51:51.377 UTC [390] WARNING: there is no transaction in progress
2026-09-29 10:51:51.394 UTC [390] WARNING: there is no transaction in progress
2026-09-29 10:51:51.412 UTC [390] WARNING: there is no transaction in progress
2026-09-29 10:51:51.427 UTC [390] WARNING: there is no transaction in progress
2026-09-29 10:51:51.443 UTC [390] WARNING: there is no transaction in progress
2026-09-29 10:51:51.466 UTC [390] WARNING: there is no transaction in progress
2026-09-29 10:51:51.520 UTC [390] WARNING: there is no transaction in progress
2026-09-29 10:51:51.526 UTC [390] WARNING: there is no transaction in progress
2026-09-29 10:51:51.541 UTC [390] WARNING: there is no transaction in progress
2026-09-29 10:51:51.551 UTC [390] WARNING: there is no transaction in progress
2026-09-29 10:51:51.561 UTC [390] WARNING: there is no transaction in progress
2026-09-29 10:51:51.575 UTC [390] WARNING: there is no transaction in progress
2026-09-29 10:51:51.585 UTC [390] WARNING: there is no transaction in progress
2026-09-29 10:51:51.591 UTC [390] WARNING: there is no transaction in progress
2026-09-29 10:51:51.598 UTC [390] WARNING: there is no transaction in progress
2026-09-29 10:51:51.604 UTC [390] WARNING: there is no transaction in progress
2026-09-29 10:51:52,275 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-29 10:51:52,275 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:48009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-29 10:51:52,520 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-29 10:51:52,521 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:48009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 10:51:52,787 [I] Table x_portal_user already exists in database 'ranger'
2026-09-29 10:51:52,787 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:48009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 10:51:53,055 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-29 10:51:53,055 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:48009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 10:51:53,311 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:48009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 10:51:53,576 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:48009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-vqch.c.gcp-upstream.internal';"
2026-09-29 10:51:53,831 [I] CORE_DB_SCHEMA import status has been updated
2026-09-29 10:51:53,832 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:48009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-29 10:51:54,063 [I] DB_PATCHES have already been applied
I20260929 10:51:54.069541 20803 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ranger-admin:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskfQc4Oq/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260929 10:51:54.069607 20803 mini_ranger.cc:193] Using host: 127.20.80.212
I20260929 10:51:54.072738 20803 mini_ranger.cc:227] Ranger admin URL: http://127.20.80.212:41317
Sep 29, 2026 10:51:54 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 10:51:54 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 10:51:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ranger-admin/ews/webapp
Sep 29, 2026 10:51:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 29, 2026 10:51:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 29, 2026 10:51:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ranger-admin/ews/webapp].
Sep 29, 2026 10:51:54 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260929 10:51:55.188300 20803 mini_ranger.cc:160] Time spent starting Ranger: real 8.625s user 0.004s sys 0.005s
Sep 29, 2026 10:52:15 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 29, 2026 10:52:15 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Sep 29, 2026 10:52:15 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 29, 2026 10:52:15 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 29, 2026 10:52:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 29, 2026 10:52:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 29, 2026 10:52:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 29, 2026 10:52:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 29, 2026 10:52:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 29, 2026 10:52:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 29, 2026 10:52:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 29, 2026 10:52:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 29, 2026 10:52:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 29, 2026 10:52:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 29, 2026 10:52:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 29, 2026 10:52:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 29, 2026 10:52:15 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 29, 2026 10:52:16 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260929 10:52:17.280646 20803 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.20.80.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.20.80.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.20.80.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/rangerkms_127.20.80.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.20.80.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/rangerkms_127.20.80.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.80.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.80.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.80.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/HTTP_127.20.80.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.80.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/HTTP_127.20.80.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 29 10:52:17 dist-test-slave-vqch krb5kdc[32643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679137, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260929 10:52:17.367302 20803 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260929 10:52:17.376992 20803 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260929 10:52:18.304103 20803 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260929 10:52:18.304188 20803 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ranger-kms
I20260929 10:52:18.304209 20803 mini_ranger_kms.cc:227] Using postgres at 127.20.80.212:48009
2026-09-29 10:52:18,464 [I] DB FLAVOR :POSTGRES
2026-09-29 10:52:18,465 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 10:52:18,465 [I] Checking connection
2026-09-29 10:52:18,465 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:48009/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-29 10:52:18,700 [I] connection success
2026-09-29 10:52:18,700 [I] --------- Verifying Ranger DB tables ---------
2026-09-29 10:52:18,700 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-29 10:52:18,700 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:48009/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-29 10:52:18,960 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-29 10:52:18,960 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 10:52:18,960 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-29 10:52:18,960 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.80.212:48009/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-29 10:52:19,229 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260929 10:52:19.719625 20803 mini_ranger_kms.cc:310] Created kms service
I20260929 10:52:19.908598 20803 mini_ranger_kms.cc:326] Created kudu user
I20260929 10:52:19.956581 20803 mini_ranger_kms.cc:343] Created rangerkms user
Sep 29 10:52:19 dist-test-slave-vqch krb5kdc[32643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679137, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.20.80.212@KRBTEST.COM
I20260929 10:52:20.371125 20803 mini_ranger_kms.cc:384] Added ranger policy
I20260929 10:52:20.371361 20803 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ranger-kms:/tmp/dist-test-taskfQc4Oq/build/release/bin/postgresql.jar:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskfQc4Oq/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260929 10:52:20.371387 20803 mini_ranger_kms.cc:240] Using host: 127.20.80.212
I20260929 10:52:20.375334 20803 mini_ranger_kms.cc:276] Ranger KMS PID: 572
I20260929 10:52:20.375406 20803 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.20.80.212:58281
Sep 29, 2026 10:52:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 10:52:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 10:52:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 29, 2026 10:52:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 29, 2026 10:52:22 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ranger-kms/ews/webapp].
Sep 29, 2026 10:52:22 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260929 10:52:22.789681 20803 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.422s user 0.010s sys 0.133s
I20260929 10:52:22.789814 20803 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260929 10:52:22.789889 20803 mini_ranger_kms.cc:401] 127.20.80.212:58281/kms/v1/keys
Sep 29 10:52:22 dist-test-slave-vqch krb5kdc[32643](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790679142, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.80.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 29 10:52:23 dist-test-slave-vqch krb5kdc[32643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679142, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.80.212@KRBTEST.COM for HTTP/127.20.80.212@KRBTEST.COM
Sep 29 10:52:23 dist-test-slave-vqch krb5kdc[32643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679142, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.80.212@KRBTEST.COM for HTTP/127.20.80.212@KRBTEST.COM
Sep 29 10:52:25 dist-test-slave-vqch krb5kdc[32643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.20.80.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
I20260929 10:52:25.957322 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:34443
--webserver_interface=127.20.80.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:38127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:34443
--ranger_config_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.80.212:58281/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:52:26.039733 623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:52:26.039963 623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:52:26.039987 623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:52:26.041247 623 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:52:26.041289 623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:52:26.041303 623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:52:26.041316 623 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:52:26.041328 623 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:52:26.042953 623 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:38127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/master-0/wal
--ranger_kms_url=127.20.80.212:58281/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:34443
--ranger_config_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:34443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:52:26.043205 623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:52:26.043448 623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:52:26.048321 629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:26.048334 631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:26.048336 628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:26.048656 623 server_base.cc:1034] running on GCE node
I20260929 10:52:26.049032 623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:52:26.049376 623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:52:26.050557 623 hybrid_clock.cc:648] HybridClock initialized: now 1790679146050543 us; error 31 us; skew 500 ppm
Sep 29 10:52:26 dist-test-slave-vqch krb5kdc[32643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679146, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:52:26.058401 623 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:52:26.058966 623 webserver.cc:460] Webserver started at http://127.20.80.254:38611/ using document root <none> and password file <none>
I20260929 10:52:26.059219 623 fs_manager.cc:359] Metadata directory not provided
I20260929 10:52:26.059298 623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:52:26.059484 623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 10:52:26 dist-test-slave-vqch krb5kdc[32643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679146, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for HTTP/127.20.80.212@KRBTEST.COM
I20260929 10:52:26.106997 623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "cb6095ecfe6241c8b8f5d23cf59acbd5"
format_stamp: "Formatted at 2026-09-29 10:52:26 on dist-test-slave-vqch"
server_key: "d8beb2911bdec3b87de31a33af1296d4"
server_key_iv: "f32576ffe9c557f16e55357f5a6b85aa"
server_key_version: "kuduclusterkey@0"
I20260929 10:52:26.107482 623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "cb6095ecfe6241c8b8f5d23cf59acbd5"
format_stamp: "Formatted at 2026-09-29 10:52:26 on dist-test-slave-vqch"
server_key: "d8beb2911bdec3b87de31a33af1296d4"
server_key_iv: "f32576ffe9c557f16e55357f5a6b85aa"
server_key_version: "kuduclusterkey@0"
I20260929 10:52:26.109349 623 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260929 10:52:26.121145 638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:26.121395 623 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 10:52:26.121497 623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "cb6095ecfe6241c8b8f5d23cf59acbd5"
format_stamp: "Formatted at 2026-09-29 10:52:26 on dist-test-slave-vqch"
server_key: "d8beb2911bdec3b87de31a33af1296d4"
server_key_iv: "f32576ffe9c557f16e55357f5a6b85aa"
server_key_version: "kuduclusterkey@0"
I20260929 10:52:26.121604 623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:52:26.132192 623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:52:26.134420 623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:52:26.134567 623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:52:26.139377 623 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:34443
I20260929 10:52:26.139447 691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:34443 every 8 connection(s)
I20260929 10:52:26.139822 623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:52:26.140558 692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:52:26.142940 692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5: Bootstrap starting.
I20260929 10:52:26.143302 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 623
I20260929 10:52:26.143417 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:52:26.143586 20803 external_mini_cluster.cc:1428] Setting key f29498bb31f4e99257c930198538bcfe
I20260929 10:52:26.143740 692 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5: Neither blocks nor log segments found. Creating new log.
I20260929 10:52:26.144037 692 log.cc:824] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5: Log is configured to *not* fsync() on all Append() calls
I20260929 10:52:26.144927 692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5: No bootstrap required, opened a new log
I20260929 10:52:26.147521 692 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb6095ecfe6241c8b8f5d23cf59acbd5" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 34443 } }
I20260929 10:52:26.147696 692 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:52:26.147725 692 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb6095ecfe6241c8b8f5d23cf59acbd5, State: Initialized, Role: FOLLOWER
I20260929 10:52:26.147877 692 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb6095ecfe6241c8b8f5d23cf59acbd5" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 34443 } }
I20260929 10:52:26.147971 692 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:52:26.148033 692 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:52:26.148108 692 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:52:26.148865 692 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb6095ecfe6241c8b8f5d23cf59acbd5" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 34443 } }
I20260929 10:52:26.149003 692 leader_election.cc:302] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb6095ecfe6241c8b8f5d23cf59acbd5; no voters:
I20260929 10:52:26.149192 692 leader_election.cc:288] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:52:26.149255 697 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:52:26.149482 697 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [term 1 LEADER]: Becoming Leader. State: Replica: cb6095ecfe6241c8b8f5d23cf59acbd5, State: Running, Role: LEADER
I20260929 10:52:26.149555 692 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:52:26.149617 697 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb6095ecfe6241c8b8f5d23cf59acbd5" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 34443 } }
I20260929 10:52:26.150097 697 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb6095ecfe6241c8b8f5d23cf59acbd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb6095ecfe6241c8b8f5d23cf59acbd5" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 34443 } } }
I20260929 10:52:26.150234 697 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [sys.catalog]: This master's current role is: LEADER
I20260929 10:52:26.150408 697 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb6095ecfe6241c8b8f5d23cf59acbd5. Latest consensus state: current_term: 1 leader_uuid: "cb6095ecfe6241c8b8f5d23cf59acbd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb6095ecfe6241c8b8f5d23cf59acbd5" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 34443 } } }
I20260929 10:52:26.150503 697 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [sys.catalog]: This master's current role is: LEADER
I20260929 10:52:26.151173 692 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Sep 29 10:52:26 dist-test-slave-vqch krb5kdc[32643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:52:26.153909 706 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:52:26.155009 706 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:52:26.159053 706 catalog_manager.cc:1347] Generated new cluster ID: 42f8986d41db4284872cb10e1389c650
I20260929 10:52:26.159096 706 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:52:26.159124 696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:26.144655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57888 (local address 127.20.80.254:34443)
0929 10:52:26.144826 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:26.144836 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:26.144872 (+ 36us) server_negotiation.cc:407] Connection header received
0929 10:52:26.145460 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:26.145469 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:26.145537 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:26.145698 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:26.146480 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:26.147913 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:26.149077 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:26.149719 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:26.149843 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:26.153341 (+ 3498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:26.153410 (+ 69us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:26.153411 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:26.153512 (+ 101us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:26.156830 (+ 3318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:26.157363 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:26.157366 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:26.157366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:26.157484 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:26.157720 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:26.157721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:26.157722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:26.158074 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:26.158650 (+ 576us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:26.158732 (+ 82us) server_negotiation.cc:299] Negotiation successful
0929 10:52:26.158894 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":76,"threads_started":1}
I20260929 10:52:26.164712 706 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:52:26.165433 706 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:52:26.169253 706 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5: Generated new TSK 0
I20260929 10:52:26.169484 706 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.80.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
I20260929 10:52:26.908763 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.80.212:58281/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.193:0
--local_ip_for_outbound_sockets=127.20.80.193
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:34443
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:38127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:52:27.014952 737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:52:27.015195 737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:52:27.015235 737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:52:27.016433 737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:52:27.016503 737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:52:27.016552 737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.193
I20260929 10:52:27.018208 737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:38127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.20.80.212:58281/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:34443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.737
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.193
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:52:27.018447 737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:52:27.018740 737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:52:27.024828 745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:27.024842 743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:27.024828 742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:27.026203 737 server_base.cc:1034] running on GCE node
I20260929 10:52:27.026432 737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:52:27.026728 737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:52:27.027916 737 hybrid_clock.cc:648] HybridClock initialized: now 1790679147027892 us; error 36 us; skew 500 ppm
Sep 29 10:52:27 dist-test-slave-vqch krb5kdc[32643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679147, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:52:27.031365 737 init.cc:377] Logged in from keytab as kudu/127.20.80.193@KRBTEST.COM (short username kudu)
I20260929 10:52:27.031912 737 webserver.cc:460] Webserver started at http://127.20.80.193:39745/ using document root <none> and password file <none>
I20260929 10:52:27.032171 737 fs_manager.cc:359] Metadata directory not provided
I20260929 10:52:27.032233 737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:52:27.032369 737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 10:52:27 dist-test-slave-vqch krb5kdc[32643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679147, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for HTTP/127.20.80.212@KRBTEST.COM
I20260929 10:52:27.045271 737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-0/data/instance:
uuid: "3b7f4f8e0bb84d31a6119ec0e9085f78"
format_stamp: "Formatted at 2026-09-29 10:52:27 on dist-test-slave-vqch"
server_key: "51308d9bdbca4f6cad53129da09fc040"
server_key_iv: "731aa754926d17ef66b640df2e0aeafe"
server_key_version: "kuduclusterkey@0"
I20260929 10:52:27.045811 737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance:
uuid: "3b7f4f8e0bb84d31a6119ec0e9085f78"
format_stamp: "Formatted at 2026-09-29 10:52:27 on dist-test-slave-vqch"
server_key: "51308d9bdbca4f6cad53129da09fc040"
server_key_iv: "731aa754926d17ef66b640df2e0aeafe"
server_key_version: "kuduclusterkey@0"
I20260929 10:52:27.049371 737 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260929 10:52:27.064997 753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:27.065258 737 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:52:27.065353 737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-0/wal
uuid: "3b7f4f8e0bb84d31a6119ec0e9085f78"
format_stamp: "Formatted at 2026-09-29 10:52:27 on dist-test-slave-vqch"
server_key: "51308d9bdbca4f6cad53129da09fc040"
server_key_iv: "731aa754926d17ef66b640df2e0aeafe"
server_key_version: "kuduclusterkey@0"
I20260929 10:52:27.065451 737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:52:27.079245 737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:52:27.081466 737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:52:27.081638 737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:52:27.081887 737 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:52:27.082304 737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:52:27.082396 737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:27.082504 737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:52:27.082584 737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:27.088789 737 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.193:33731
I20260929 10:52:27.089588 737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
I20260929 10:52:27.091730 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 737
I20260929 10:52:27.091827 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance
I20260929 10:52:27.092028 20803 external_mini_cluster.cc:1428] Setting key 7b1aa7b1f1e06546877938b78ab5ea6a
I20260929 10:52:27.096958 867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.193:33731 every 8 connection(s)
Sep 29 10:52:27 dist-test-slave-vqch krb5kdc[32643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679147, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.80.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.194@KRBTEST.COM" created.
I20260929 10:52:27.113683 870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:27.091501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:34443 (local address 127.20.80.193:39799)
0929 10:52:27.097103 (+ 5602us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:27.097152 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:27.097804 (+ 652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:27.098254 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:27.098259 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:27.098499 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:52:27.098798 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:27.098805 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:27.099829 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:27.099830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:27.100639 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:27.100641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:27.101179 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:27.101180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:27.101325 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:27.102014 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:52:27.102024 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:52:27.103734 (+ 1710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:52:27.111783 (+ 8049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:27.111796 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:27.111800 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:27.112199 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:27.112471 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:27.112472 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:27.112473 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:27.112629 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:27.112926 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:52:27.112932 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:52:27.113135 (+ 203us) client_negotiation.cc:769] Sending connection context
0929 10:52:27.113247 (+ 112us) client_negotiation.cc:240] Negotiation successful
0929 10:52:27.113429 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5553,"thread_start_us":85,"threads_started":1}
I20260929 10:52:27.114533 871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:27.091523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:39799 (local address 127.20.80.254:34443)
0929 10:52:27.097998 (+ 6475us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:27.098001 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:27.098008 (+ 7us) server_negotiation.cc:407] Connection header received
0929 10:52:27.098033 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:27.098033 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:27.098045 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:27.098108 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:27.098914 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:27.099718 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:27.100743 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:27.101062 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:27.107430 (+ 6368us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:27.107497 (+ 67us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:27.107498 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:27.107499 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:27.107505 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:27.109686 (+ 2181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:27.112304 (+ 2618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:27.112305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:27.112306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:27.112375 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:27.112708 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:27.112709 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:27.112709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:27.112853 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:27.114294 (+ 1441us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:27.114338 (+ 44us) server_negotiation.cc:299] Negotiation successful
0929 10:52:27.114435 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6443,"thread_start_us":87,"threads_started":1}
I20260929 10:52:27.115105 868 heartbeater.cc:347] Connected to a master server at 127.20.80.254:34443
I20260929 10:52:27.115222 868 heartbeater.cc:464] Registering TS with master...
I20260929 10:52:27.115424 868 heartbeater.cc:511] Master 127.20.80.254:34443 requested a full tablet report, sending...
I20260929 10:52:27.116014 655 ts_manager.cc:194] Registered new tserver with Master: 3b7f4f8e0bb84d31a6119ec0e9085f78 (127.20.80.193:33731)
I20260929 10:52:27.116936 655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.193@KRBTEST.COM'} at 127.20.80.193:39799
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
I20260929 10:52:27.136678 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.80.212:58281/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.194:0
--local_ip_for_outbound_sockets=127.20.80.194
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:34443
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:38127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 10:52:27 dist-test-slave-vqch krb5kdc[32643](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790679147, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260929 10:52:27.262395 874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:52:27.262674 874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:52:27.262717 874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:52:27.264468 874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:52:27.264559 874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:52:27.264618 874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.194
I20260929 10:52:27.266835 874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:38127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.20.80.212:58281/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:34443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.194
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:52:27.267091 874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:52:27.267379 874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:52:27.273764 882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:27.273764 881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:27.273855 884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:27.274220 874 server_base.cc:1034] running on GCE node
I20260929 10:52:27.274435 874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:52:27.274690 874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:52:27.275862 874 hybrid_clock.cc:648] HybridClock initialized: now 1790679147275841 us; error 32 us; skew 500 ppm
Sep 29 10:52:27 dist-test-slave-vqch krb5kdc[32643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679147, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:52:27.279069 874 init.cc:377] Logged in from keytab as kudu/127.20.80.194@KRBTEST.COM (short username kudu)
I20260929 10:52:27.279621 874 webserver.cc:460] Webserver started at http://127.20.80.194:33019/ using document root <none> and password file <none>
I20260929 10:52:27.279917 874 fs_manager.cc:359] Metadata directory not provided
I20260929 10:52:27.280015 874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:52:27.280184 874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 10:52:27 dist-test-slave-vqch krb5kdc[32643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679147, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for HTTP/127.20.80.212@KRBTEST.COM
I20260929 10:52:27.302340 874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-1/data/instance:
uuid: "2189787f754d44b59de47ecebf6ab8ea"
format_stamp: "Formatted at 2026-09-29 10:52:27 on dist-test-slave-vqch"
server_key: "62a016d38215cce7bd18e46180d06b20"
server_key_iv: "e826d9def54248a9be49e6ed52ad64c1"
server_key_version: "kuduclusterkey@0"
I20260929 10:52:27.302776 874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance:
uuid: "2189787f754d44b59de47ecebf6ab8ea"
format_stamp: "Formatted at 2026-09-29 10:52:27 on dist-test-slave-vqch"
server_key: "62a016d38215cce7bd18e46180d06b20"
server_key_iv: "e826d9def54248a9be49e6ed52ad64c1"
server_key_version: "kuduclusterkey@0"
I20260929 10:52:27.304755 874 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260929 10:52:27.318871 891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:27.319779 874 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 10:52:27.319908 874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-1/wal
uuid: "2189787f754d44b59de47ecebf6ab8ea"
format_stamp: "Formatted at 2026-09-29 10:52:27 on dist-test-slave-vqch"
server_key: "62a016d38215cce7bd18e46180d06b20"
server_key_iv: "e826d9def54248a9be49e6ed52ad64c1"
server_key_version: "kuduclusterkey@0"
I20260929 10:52:27.320026 874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:52:27.337126 874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:52:27.339732 874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:52:27.339897 874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:52:27.340158 874 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:52:27.340584 874 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:52:27.340639 874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:27.340706 874 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:52:27.340744 874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:27.347577 874 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.194:37081
I20260929 10:52:27.347846 1007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.194:37081 every 8 connection(s)
I20260929 10:52:27.348083 874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
I20260929 10:52:27.349853 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 874
I20260929 10:52:27.349951 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance
I20260929 10:52:27.350160 20803 external_mini_cluster.cc:1428] Setting key 488a3cf9a83fe6cd9732ce4baafa410a
Sep 29 10:52:27 dist-test-slave-vqch krb5kdc[32643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679147, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:52:27.364077 871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:27.349132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:48135 (local address 127.20.80.254:34443)
0929 10:52:27.349178 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:27.349180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:27.353268 (+ 4088us) server_negotiation.cc:407] Connection header received
0929 10:52:27.353971 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:27.353972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:27.354006 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:27.354037 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:27.354948 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:27.355746 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:27.357299 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:27.357597 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:27.357642 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:27.360473 (+ 2831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:27.360474 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:27.360475 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:27.360483 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:27.362324 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:27.362908 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:27.362909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:27.362909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:27.362972 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:27.363258 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:27.363259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:27.363259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:27.363436 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:27.363483 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:27.363874 (+ 391us) server_negotiation.cc:299] Negotiation successful
0929 10:52:27.363971 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 10:52:27.364856 1010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:27.348845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:34443 (local address 127.20.80.194:48135)
0929 10:52:27.353125 (+ 4280us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:27.353181 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:27.353850 (+ 669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:27.354286 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:27.354290 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:27.354535 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:52:27.354825 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:27.354832 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:27.355903 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:27.355904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:27.357195 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:27.357198 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:27.357778 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:27.357779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:27.357920 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:27.358596 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:52:27.358606 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:52:27.360352 (+ 1746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:52:27.362435 (+ 2083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:27.362441 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:27.362443 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:27.362798 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:27.363045 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:27.363046 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:27.363046 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:27.363175 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:27.363535 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:52:27.363542 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:52:27.363716 (+ 174us) client_negotiation.cc:769] Sending connection context
0929 10:52:27.364441 (+ 725us) client_negotiation.cc:240] Negotiation successful
0929 10:52:27.364604 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4227,"thread_start_us":74,"threads_started":1}
I20260929 10:52:27.365615 1008 heartbeater.cc:347] Connected to a master server at 127.20.80.254:34443
I20260929 10:52:27.365729 1008 heartbeater.cc:464] Registering TS with master...
I20260929 10:52:27.365913 1008 heartbeater.cc:511] Master 127.20.80.254:34443 requested a full tablet report, sending...
I20260929 10:52:27.366312 655 ts_manager.cc:194] Registered new tserver with Master: 2189787f754d44b59de47ecebf6ab8ea (127.20.80.194:37081)
I20260929 10:52:27.366813 655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.194@KRBTEST.COM'} at 127.20.80.194:48135
WARNING: no policy specified for kudu/127.20.80.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
I20260929 10:52:27.400642 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.80.212:58281/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.195:0
--local_ip_for_outbound_sockets=127.20.80.195
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:34443
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:38127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:52:27.513985 1013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:52:27.514312 1013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:52:27.514364 1013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:52:27.516040 1013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:52:27.516098 1013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:52:27.516139 1013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.195
I20260929 10:52:27.518316 1013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:38127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.20.80.212:58281/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:34443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.195
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:52:27.518738 1013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:52:27.519042 1013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:52:27.525230 1021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:27.528257 1019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:27.528352 1018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:27.528512 1013 server_base.cc:1034] running on GCE node
I20260929 10:52:27.528750 1013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:52:27.529062 1013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:52:27.530246 1013 hybrid_clock.cc:648] HybridClock initialized: now 1790679147530189 us; error 81 us; skew 500 ppm
Sep 29 10:52:27 dist-test-slave-vqch krb5kdc[32643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679147, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:52:27.533730 1013 init.cc:377] Logged in from keytab as kudu/127.20.80.195@KRBTEST.COM (short username kudu)
I20260929 10:52:27.534344 1013 webserver.cc:460] Webserver started at http://127.20.80.195:36813/ using document root <none> and password file <none>
I20260929 10:52:27.534677 1013 fs_manager.cc:359] Metadata directory not provided
I20260929 10:52:27.534808 1013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:52:27.534994 1013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 10:52:27 dist-test-slave-vqch krb5kdc[32643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679147, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for HTTP/127.20.80.212@KRBTEST.COM
I20260929 10:52:27.549032 1013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-2/data/instance:
uuid: "68a8f7cfde144e13a79cac6ba73aaecc"
format_stamp: "Formatted at 2026-09-29 10:52:27 on dist-test-slave-vqch"
server_key: "57838f82e7c96bd3613c97ab4484d5fe"
server_key_iv: "10e4b55e89e3045619419a6332d9dc00"
server_key_version: "kuduclusterkey@0"
I20260929 10:52:27.549626 1013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance:
uuid: "68a8f7cfde144e13a79cac6ba73aaecc"
format_stamp: "Formatted at 2026-09-29 10:52:27 on dist-test-slave-vqch"
server_key: "57838f82e7c96bd3613c97ab4484d5fe"
server_key_iv: "10e4b55e89e3045619419a6332d9dc00"
server_key_version: "kuduclusterkey@0"
I20260929 10:52:27.551528 1013 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260929 10:52:27.565024 1029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:27.565246 1013 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:52:27.565357 1013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-2/wal
uuid: "68a8f7cfde144e13a79cac6ba73aaecc"
format_stamp: "Formatted at 2026-09-29 10:52:27 on dist-test-slave-vqch"
server_key: "57838f82e7c96bd3613c97ab4484d5fe"
server_key_iv: "10e4b55e89e3045619419a6332d9dc00"
server_key_version: "kuduclusterkey@0"
I20260929 10:52:27.565469 1013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:52:27.608367 1013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:52:27.611274 1013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:52:27.611500 1013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:52:27.611797 1013 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:52:27.612205 1013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:52:27.612300 1013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:27.612385 1013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:52:27.612461 1013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:27.620355 1013 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.195:33103
I20260929 10:52:27.620509 1143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.195:33103 every 8 connection(s)
I20260929 10:52:27.621042 1013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
I20260929 10:52:27.629220 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 1013
Sep 29 10:52:27 dist-test-slave-vqch krb5kdc[32643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679147, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:52:27.629334 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance
I20260929 10:52:27.629534 20803 external_mini_cluster.cc:1428] Setting key 7da9a5a8cde341f94b16bd816eaeffd4
I20260929 10:52:27.634343 871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:27.621913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:50559 (local address 127.20.80.254:34443)
0929 10:52:27.621963 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:27.621965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:27.622504 (+ 539us) server_negotiation.cc:407] Connection header received
0929 10:52:27.623049 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:27.623051 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:27.623064 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:27.623134 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:27.623914 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:27.624781 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:27.626494 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:27.626895 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:27.626943 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:27.629922 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:27.629923 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:27.629924 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:27.629932 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:27.632140 (+ 2208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:27.633073 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:27.633074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:27.633075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:27.633146 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:27.633511 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:27.633512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:27.633512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:27.633660 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:27.633750 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:27.634143 (+ 393us) server_negotiation.cc:299] Negotiation successful
0929 10:52:27.634235 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 10:52:27.634567 1146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:27.621920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:34443 (local address 127.20.80.195:50559)
0929 10:52:27.622138 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:27.622185 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:27.622918 (+ 733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:27.623206 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:27.623210 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:27.623499 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:52:27.623798 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:27.623804 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:27.624934 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:27.624935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:27.626371 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:27.626385 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:27.627081 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:27.627082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:27.627249 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:27.627918 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:52:27.627928 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:52:27.629615 (+ 1687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:52:27.632268 (+ 2653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:27.632277 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:27.632280 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:27.632933 (+ 653us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:27.633256 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:27.633256 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:27.633257 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:27.633405 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:27.633765 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:52:27.633774 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:52:27.633943 (+ 169us) client_negotiation.cc:769] Sending connection context
0929 10:52:27.634071 (+ 128us) client_negotiation.cc:240] Negotiation successful
0929 10:52:27.634248 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":103,"threads_started":1}
I20260929 10:52:27.635161 1144 heartbeater.cc:347] Connected to a master server at 127.20.80.254:34443
I20260929 10:52:27.635272 1144 heartbeater.cc:464] Registering TS with master...
I20260929 10:52:27.635474 1144 heartbeater.cc:511] Master 127.20.80.254:34443 requested a full tablet report, sending...
I20260929 10:52:27.635879 655 ts_manager.cc:194] Registered new tserver with Master: 68a8f7cfde144e13a79cac6ba73aaecc (127.20.80.195:33103)
I20260929 10:52:27.636396 655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.195@KRBTEST.COM'} at 127.20.80.195:50559
I20260929 10:52:27.641917 20803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:52:27.642619 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 737
I20260929 10:52:27.648765 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 874
I20260929 10:52:27.653822 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 1013
I20260929 10:52:27.659952 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 623
I20260929 10:52:27.665980 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:34443
--webserver_interface=127.20.80.254
--webserver_port=38611
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:38127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:34443
--ranger_config_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.80.212:58281/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.80.212:58281/kms with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:52:27.747478 1147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:52:27.747696 1147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:52:27.747718 1147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:52:27.748891 1147 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:52:27.748936 1147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:52:27.748950 1147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:52:27.749035 1147 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:52:27.749068 1147 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:52:27.750636 1147 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:38127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.80.212:58281/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:34443
--ranger_config_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:34443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=38611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:52:27.750895 1147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:52:27.751139 1147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:52:27.756214 1153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:27.756234 1155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:27.756224 1152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:27.756477 1147 server_base.cc:1034] running on GCE node
I20260929 10:52:27.756735 1147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:52:27.757035 1147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:52:27.758188 1147 hybrid_clock.cc:648] HybridClock initialized: now 1790679147758164 us; error 41 us; skew 500 ppm
Sep 29 10:52:27 dist-test-slave-vqch krb5kdc[32643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679147, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:52:27.760766 1147 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:52:27.761226 1147 webserver.cc:460] Webserver started at http://127.20.80.254:38611/ using document root <none> and password file <none>
I20260929 10:52:27.761425 1147 fs_manager.cc:359] Metadata directory not provided
I20260929 10:52:27.761472 1147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 10:52:27 dist-test-slave-vqch krb5kdc[32643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679147, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for HTTP/127.20.80.212@KRBTEST.COM
I20260929 10:52:27.813150 1147 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260929 10:52:27.814126 1163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:27.814380 1147 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:52:27.814495 1147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "cb6095ecfe6241c8b8f5d23cf59acbd5"
format_stamp: "Formatted at 2026-09-29 10:52:26 on dist-test-slave-vqch"
server_key: "d8beb2911bdec3b87de31a33af1296d4"
server_key_iv: "f32576ffe9c557f16e55357f5a6b85aa"
server_key_version: "kuduclusterkey@0"
I20260929 10:52:27.815012 1147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:52:27.838235 1147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:52:27.840363 1147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:52:27.840531 1147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:52:27.845894 1147 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:34443
I20260929 10:52:27.845975 1215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:34443 every 8 connection(s)
I20260929 10:52:27.846372 1147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:52:27.847649 1216 sys_catalog.cc:263] Verifying existing consensus state
I20260929 10:52:27.848229 1216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5: Bootstrap starting.
I20260929 10:52:27.851989 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 1147
I20260929 10:52:27.852353 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.80.212:58281/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.193:33731
--local_ip_for_outbound_sockets=127.20.80.193
--tserver_master_addrs=127.20.80.254:34443
--webserver_port=39745
--webserver_interface=127.20.80.193
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:38127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.80.212:58281/kms with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:52:27.857345 1216 log.cc:824] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5: Log is configured to *not* fsync() on all Append() calls
I20260929 10:52:27.859807 1216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 10:52:27.860162 1216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5: Bootstrap complete.
I20260929 10:52:27.864333 1216 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb6095ecfe6241c8b8f5d23cf59acbd5" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 34443 } }
I20260929 10:52:27.864646 1216 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb6095ecfe6241c8b8f5d23cf59acbd5, State: Initialized, Role: FOLLOWER
I20260929 10:52:27.864820 1216 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb6095ecfe6241c8b8f5d23cf59acbd5" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 34443 } }
I20260929 10:52:27.864884 1216 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:52:27.864943 1216 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:52:27.865007 1216 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [term 1 FOLLOWER]: Advancing to term 2
I20260929 10:52:27.866302 1216 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb6095ecfe6241c8b8f5d23cf59acbd5" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 34443 } }
I20260929 10:52:27.866412 1216 leader_election.cc:302] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb6095ecfe6241c8b8f5d23cf59acbd5; no voters:
I20260929 10:52:27.866564 1216 leader_election.cc:288] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 10:52:27.866657 1220 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 10:52:27.866807 1220 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [term 2 LEADER]: Becoming Leader. State: Replica: cb6095ecfe6241c8b8f5d23cf59acbd5, State: Running, Role: LEADER
I20260929 10:52:27.866815 1216 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:52:27.868220 1220 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb6095ecfe6241c8b8f5d23cf59acbd5" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 34443 } }
I20260929 10:52:27.868824 1222 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb6095ecfe6241c8b8f5d23cf59acbd5. Latest consensus state: current_term: 2 leader_uuid: "cb6095ecfe6241c8b8f5d23cf59acbd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb6095ecfe6241c8b8f5d23cf59acbd5" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 34443 } } }
I20260929 10:52:27.868909 1222 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [sys.catalog]: This master's current role is: LEADER
I20260929 10:52:27.868969 1216 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260929 10:52:27.869117 1221 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "cb6095ecfe6241c8b8f5d23cf59acbd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb6095ecfe6241c8b8f5d23cf59acbd5" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 34443 } } }
I20260929 10:52:27.869174 1221 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5 [sys.catalog]: This master's current role is: LEADER
I20260929 10:52:27.869263 1235 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:52:27.871788 1235 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:52:27.872042 1235 catalog_manager.cc:1259] Loaded cluster ID: 42f8986d41db4284872cb10e1389c650
I20260929 10:52:27.872082 1235 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:52:27.873847 1235 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:52:27.874029 1235 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5: Loaded TSK: 0
I20260929 10:52:27.874246 1235 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 10:52:27.969218 1218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:52:27.969587 1218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:52:27.969676 1218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:52:27.971386 1218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:52:27.971448 1218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:52:27.971489 1218 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.193
I20260929 10:52:27.973600 1218 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:38127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.80.212:58281/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.193:33731
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.80.193
--webserver_port=39745
--tserver_master_addrs=127.20.80.254:34443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.193
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:52:27.973839 1218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:52:27.974118 1218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:52:27.983449 1257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:27.983920 1218 server_base.cc:1034] running on GCE node
W20260929 10:52:27.984092 1254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:27.984135 1255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:27.984514 1218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:52:27.984886 1218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:52:27.986066 1218 hybrid_clock.cc:648] HybridClock initialized: now 1790679147986043 us; error 36 us; skew 500 ppm
Sep 29 10:52:27 dist-test-slave-vqch krb5kdc[32643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679147, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:52:27.989238 1218 init.cc:377] Logged in from keytab as kudu/127.20.80.193@KRBTEST.COM (short username kudu)
I20260929 10:52:27.989768 1218 webserver.cc:460] Webserver started at http://127.20.80.193:39745/ using document root <none> and password file <none>
I20260929 10:52:27.990042 1218 fs_manager.cc:359] Metadata directory not provided
I20260929 10:52:27.990130 1218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 10:52:27 dist-test-slave-vqch krb5kdc[32643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679147, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for HTTP/127.20.80.212@KRBTEST.COM
I20260929 10:52:28.018528 1218 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260929 10:52:28.019882 1265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:28.020097 1218 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:52:28.020206 1218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-0/wal
uuid: "3b7f4f8e0bb84d31a6119ec0e9085f78"
format_stamp: "Formatted at 2026-09-29 10:52:27 on dist-test-slave-vqch"
server_key: "51308d9bdbca4f6cad53129da09fc040"
server_key_iv: "731aa754926d17ef66b640df2e0aeafe"
server_key_version: "kuduclusterkey@0"
I20260929 10:52:28.020750 1218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:52:28.031818 1218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:52:28.034168 1218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:52:28.034338 1218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:52:28.034615 1218 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:52:28.034970 1218 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:52:28.035040 1218 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:28.035089 1218 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:52:28.035123 1218 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:28.042033 1218 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.193:33731
I20260929 10:52:28.042560 1218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
I20260929 10:52:28.046029 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 1218
I20260929 10:52:28.046325 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.80.212:58281/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.194:37081
--local_ip_for_outbound_sockets=127.20.80.194
--tserver_master_addrs=127.20.80.254:34443
--webserver_port=33019
--webserver_interface=127.20.80.194
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:38127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.80.212:58281/kms with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:52:28.044296 1378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.193:33731 every 8 connection(s)
Sep 29 10:52:28 dist-test-slave-vqch krb5kdc[32643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679147, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:52:28.059840 1381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:28.044517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:34443 (local address 127.20.80.193:57887)
0929 10:52:28.048405 (+ 3888us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:28.048463 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:28.049044 (+ 581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:28.049611 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:28.049620 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:28.049796 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:52:28.049994 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:28.049999 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:28.051165 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:28.051166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:28.051958 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:28.051961 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:28.052912 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:28.052914 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:28.053081 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:28.053708 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:52:28.053718 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:52:28.055380 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:52:28.057940 (+ 2560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:28.057948 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:28.057950 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:28.058266 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:28.058559 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:28.058559 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:28.058560 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:28.058690 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:28.059181 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:52:28.059187 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:52:28.059345 (+ 158us) client_negotiation.cc:769] Sending connection context
0929 10:52:28.059439 (+ 94us) client_negotiation.cc:240] Negotiation successful
0929 10:52:28.059598 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3837,"thread_start_us":74,"threads_started":1}
I20260929 10:52:28.060881 1382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:28.044602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:57887 (local address 127.20.80.254:34443)
0929 10:52:28.044948 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:28.044957 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:28.048587 (+ 3630us) server_negotiation.cc:407] Connection header received
0929 10:52:28.049313 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:28.049320 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:28.049390 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:28.050097 (+ 707us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:28.050158 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:28.051040 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:28.052072 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:28.052749 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:28.052900 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:28.055530 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:28.055534 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:28.055535 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:28.055547 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:28.057834 (+ 2287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:28.058372 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:28.058375 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:28.058375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:28.058484 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:28.058772 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:28.058773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:28.058774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:28.059075 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:28.060474 (+ 1399us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:28.060553 (+ 79us) server_negotiation.cc:299] Negotiation successful
0929 10:52:28.060681 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":73,"threads_started":1}
I20260929 10:52:28.061873 1379 heartbeater.cc:347] Connected to a master server at 127.20.80.254:34443
I20260929 10:52:28.061969 1379 heartbeater.cc:464] Registering TS with master...
I20260929 10:52:28.062156 1379 heartbeater.cc:511] Master 127.20.80.254:34443 requested a full tablet report, sending...
W20260929 10:52:28.063071 1379 heartbeater.cc:644] Failed to heartbeat to 127.20.80.254:34443 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 10:52:28.063141 1379 heartbeater.cc:464] Registering TS with master...
I20260929 10:52:28.063218 1379 heartbeater.cc:511] Master 127.20.80.254:34443 requested a full tablet report, sending...
I20260929 10:52:28.063618 1379 heartbeater.cc:464] Registering TS with master...
I20260929 10:52:28.063690 1379 heartbeater.cc:511] Master 127.20.80.254:34443 requested a full tablet report, sending...
W20260929 10:52:28.064036 1379 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 10:52:28.164201 1383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:52:28.164517 1383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:52:28.164556 1383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:52:28.166549 1383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:52:28.166637 1383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:52:28.166683 1383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.194
I20260929 10:52:28.169013 1383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:38127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.80.212:58281/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.194:37081
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.80.194
--webserver_port=33019
--tserver_master_addrs=127.20.80.254:34443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.194
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:52:28.169274 1383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:52:28.169539 1383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:52:28.177785 1390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:28.177789 1389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:28.179204 1392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:28.179833 1383 server_base.cc:1034] running on GCE node
I20260929 10:52:28.180074 1383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:52:28.180368 1383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:52:28.181528 1383 hybrid_clock.cc:648] HybridClock initialized: now 1790679148181517 us; error 27 us; skew 500 ppm
Sep 29 10:52:28 dist-test-slave-vqch krb5kdc[32643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679148, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:52:28.184201 1383 init.cc:377] Logged in from keytab as kudu/127.20.80.194@KRBTEST.COM (short username kudu)
I20260929 10:52:28.184755 1383 webserver.cc:460] Webserver started at http://127.20.80.194:33019/ using document root <none> and password file <none>
I20260929 10:52:28.185071 1383 fs_manager.cc:359] Metadata directory not provided
I20260929 10:52:28.185158 1383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 10:52:28 dist-test-slave-vqch krb5kdc[32643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679148, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for HTTP/127.20.80.212@KRBTEST.COM
I20260929 10:52:28.199486 1383 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260929 10:52:28.200567 1400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:28.200791 1383 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:52:28.200901 1383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-1/wal
uuid: "2189787f754d44b59de47ecebf6ab8ea"
format_stamp: "Formatted at 2026-09-29 10:52:27 on dist-test-slave-vqch"
server_key: "62a016d38215cce7bd18e46180d06b20"
server_key_iv: "e826d9def54248a9be49e6ed52ad64c1"
server_key_version: "kuduclusterkey@0"
I20260929 10:52:28.201406 1383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:52:28.226296 1383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:52:28.228767 1383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:52:28.229022 1383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:52:28.229307 1383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:52:28.229671 1383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:52:28.229729 1383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:28.229763 1383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:52:28.229789 1383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:28.236588 1383 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.194:37081
I20260929 10:52:28.237129 1383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
I20260929 10:52:28.237104 1513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.194:37081 every 8 connection(s)
I20260929 10:52:28.237725 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 1383
I20260929 10:52:28.238024 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.80.212:58281/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.195:33103
--local_ip_for_outbound_sockets=127.20.80.195
--tserver_master_addrs=127.20.80.254:34443
--webserver_port=36813
--webserver_interface=127.20.80.195
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:38127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.80.212:58281/kms with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 10:52:28 dist-test-slave-vqch krb5kdc[32643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679148, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:52:28.250829 1382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:28.240049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:60423 (local address 127.20.80.254:34443)
0929 10:52:28.240102 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:28.240103 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:28.240354 (+ 251us) server_negotiation.cc:407] Connection header received
0929 10:52:28.241106 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:28.241108 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:28.241120 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:28.241152 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:28.241975 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:28.242728 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:28.244207 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:28.244505 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:28.244550 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:28.247231 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:28.247232 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:28.247233 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:28.247240 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:28.249134 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:28.249648 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:28.249649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:28.249650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:28.249716 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:28.250057 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:28.250059 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:28.250059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:28.250198 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:28.250244 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:28.250640 (+ 396us) server_negotiation.cc:299] Negotiation successful
0929 10:52:28.250732 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 10:52:28.251396 1517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:28.240038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:34443 (local address 127.20.80.194:60423)
0929 10:52:28.240213 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:28.240261 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:28.240952 (+ 691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:28.241339 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:28.241343 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:28.241575 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:52:28.241842 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:28.241848 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:28.242844 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:28.242845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:28.244109 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:28.244111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:28.244667 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:28.244668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:28.244805 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:28.245449 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:52:28.245458 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:52:28.247101 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:52:28.249225 (+ 2124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:28.249232 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:28.249235 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:28.249550 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:28.249800 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:28.249801 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:28.249801 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:28.249957 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:28.250295 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:52:28.250303 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:52:28.250474 (+ 171us) client_negotiation.cc:769] Sending connection context
0929 10:52:28.250991 (+ 517us) client_negotiation.cc:240] Negotiation successful
0929 10:52:28.251160 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
I20260929 10:52:28.251870 1514 heartbeater.cc:347] Connected to a master server at 127.20.80.254:34443
I20260929 10:52:28.251978 1514 heartbeater.cc:464] Registering TS with master...
I20260929 10:52:28.252172 1514 heartbeater.cc:511] Master 127.20.80.254:34443 requested a full tablet report, sending...
W20260929 10:52:28.252813 1514 heartbeater.cc:644] Failed to heartbeat to 127.20.80.254:34443 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 10:52:28.252904 1514 heartbeater.cc:464] Registering TS with master...
I20260929 10:52:28.253001 1514 heartbeater.cc:511] Master 127.20.80.254:34443 requested a full tablet report, sending...
I20260929 10:52:28.253369 1514 heartbeater.cc:464] Registering TS with master...
I20260929 10:52:28.253448 1514 heartbeater.cc:511] Master 127.20.80.254:34443 requested a full tablet report, sending...
W20260929 10:52:28.253770 1514 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 10:52:28.343919 1516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:52:28.344226 1516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:52:28.344275 1516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:52:28.345695 1516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:52:28.345769 1516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:52:28.345818 1516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.195
I20260929 10:52:28.347935 1516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:38127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.80.212:58281/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.195:33103
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.80.195
--webserver_port=36813
--tserver_master_addrs=127.20.80.254:34443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.195
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:52:28.348222 1516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:52:28.348506 1516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:52:28.355610 1526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:28.355652 1516 server_base.cc:1034] running on GCE node
W20260929 10:52:28.355885 1524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:28.357144 1523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:28.358664 1516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:52:28.358982 1516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:52:28.360149 1516 hybrid_clock.cc:648] HybridClock initialized: now 1790679148360138 us; error 34 us; skew 500 ppm
Sep 29 10:52:28 dist-test-slave-vqch krb5kdc[32643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679148, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:52:28.363072 1516 init.cc:377] Logged in from keytab as kudu/127.20.80.195@KRBTEST.COM (short username kudu)
I20260929 10:52:28.363615 1516 webserver.cc:460] Webserver started at http://127.20.80.195:36813/ using document root <none> and password file <none>
I20260929 10:52:28.363968 1516 fs_manager.cc:359] Metadata directory not provided
I20260929 10:52:28.364053 1516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 10:52:28 dist-test-slave-vqch krb5kdc[32643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679148, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for HTTP/127.20.80.212@KRBTEST.COM
I20260929 10:52:28.383060 1516 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260929 10:52:28.389106 1535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:28.389338 1516 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260929 10:52:28.389449 1516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-2/wal
uuid: "68a8f7cfde144e13a79cac6ba73aaecc"
format_stamp: "Formatted at 2026-09-29 10:52:27 on dist-test-slave-vqch"
server_key: "57838f82e7c96bd3613c97ab4484d5fe"
server_key_iv: "10e4b55e89e3045619419a6332d9dc00"
server_key_version: "kuduclusterkey@0"
I20260929 10:52:28.389931 1516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:52:28.404577 1516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:52:28.406960 1516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:52:28.407116 1516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:52:28.407380 1516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:52:28.407747 1516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:52:28.407795 1516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:28.407825 1516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:52:28.407848 1516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:28.415131 1516 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.195:33103
I20260929 10:52:28.415223 1648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.195:33103 every 8 connection(s)
I20260929 10:52:28.415566 1516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
Sep 29 10:52:28 dist-test-slave-vqch krb5kdc[32643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679148, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:52:28.424645 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 1516
I20260929 10:52:28.428622 1382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:28.417576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:52693 (local address 127.20.80.254:34443)
0929 10:52:28.418809 (+ 1233us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:28.418812 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:28.418823 (+ 11us) server_negotiation.cc:407] Connection header received
0929 10:52:28.418852 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:28.418853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:28.418873 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:28.418937 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:28.419703 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:28.420323 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:28.421853 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:28.422171 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:28.422214 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:28.425271 (+ 3057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:28.425273 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:28.425273 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:28.425281 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:28.426824 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:28.427532 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:28.427533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:28.427534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:28.427605 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:28.427920 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:28.427922 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:28.427922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:28.428063 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:28.428148 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:28.428444 (+ 296us) server_negotiation.cc:299] Negotiation successful
0929 10:52:28.428530 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1199}
I20260929 10:52:28.428810 1651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:28.417565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:34443 (local address 127.20.80.195:52693)
0929 10:52:28.417775 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:28.417824 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:28.418461 (+ 637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:28.419077 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:28.419082 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:28.419311 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:52:28.419583 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:28.419590 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:28.420439 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:28.420440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:28.421747 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:28.421751 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:28.422321 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:28.422322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:28.422471 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:28.423069 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:52:28.423079 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:52:28.424778 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:52:28.427043 (+ 2265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:28.427051 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:28.427054 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:28.427427 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:28.427689 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:28.427689 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:28.427690 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:28.427833 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:28.428148 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:52:28.428152 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:52:28.428277 (+ 125us) client_negotiation.cc:769] Sending connection context
0929 10:52:28.428399 (+ 122us) client_negotiation.cc:240] Negotiation successful
0929 10:52:28.428554 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":87,"threads_started":1}
I20260929 10:52:28.429315 1649 heartbeater.cc:347] Connected to a master server at 127.20.80.254:34443
I20260929 10:52:28.429405 1649 heartbeater.cc:464] Registering TS with master...
I20260929 10:52:28.429586 1649 heartbeater.cc:511] Master 127.20.80.254:34443 requested a full tablet report, sending...
W20260929 10:52:28.430328 1649 heartbeater.cc:644] Failed to heartbeat to 127.20.80.254:34443 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 10:52:28.430428 1649 heartbeater.cc:464] Registering TS with master...
I20260929 10:52:28.430501 1649 heartbeater.cc:511] Master 127.20.80.254:34443 requested a full tablet report, sending...
I20260929 10:52:28.430815 1649 heartbeater.cc:464] Registering TS with master...
I20260929 10:52:28.430888 1649 heartbeater.cc:511] Master 127.20.80.254:34443 requested a full tablet report, sending...
W20260929 10:52:28.431212 1649 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 10:52:28.436601 1652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:28.425088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57890 (local address 127.20.80.254:34443)
0929 10:52:28.425223 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:28.425225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:28.425465 (+ 240us) server_negotiation.cc:407] Connection header received
0929 10:52:28.425802 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:28.425804 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:28.425823 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:28.425892 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:28.427195 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:28.427982 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:28.429834 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:28.430165 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:28.430248 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:28.431732 (+ 1484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:28.431733 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:28.431733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:28.431739 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:28.434425 (+ 2686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:28.435838 (+ 1413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:28.435839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:28.435839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:28.435896 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:28.436133 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:28.436134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:28.436134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:28.436238 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:28.436362 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:28.436466 (+ 104us) server_negotiation.cc:299] Negotiation successful
0929 10:52:28.436532 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
Sep 29 10:52:28 dist-test-slave-vqch krb5kdc[32643](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790679148, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:52:29.064193 1379 heartbeater.cc:464] Registering TS with master...
I20260929 10:52:29.064365 1379 heartbeater.cc:511] Master 127.20.80.254:34443 requested a full tablet report, sending...
I20260929 10:52:29.064697 1180 ts_manager.cc:194] Registered new tserver with Master: 3b7f4f8e0bb84d31a6119ec0e9085f78 (127.20.80.193:33731)
I20260929 10:52:29.065418 1180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.193@KRBTEST.COM'} at 127.20.80.193:57887
I20260929 10:52:29.253978 1514 heartbeater.cc:464] Registering TS with master...
I20260929 10:52:29.254139 1514 heartbeater.cc:511] Master 127.20.80.254:34443 requested a full tablet report, sending...
I20260929 10:52:29.254491 1180 ts_manager.cc:194] Registered new tserver with Master: 2189787f754d44b59de47ecebf6ab8ea (127.20.80.194:37081)
I20260929 10:52:29.255029 1180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.194@KRBTEST.COM'} at 127.20.80.194:60423
Sep 29 10:52:29 dist-test-slave-vqch krb5kdc[32643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679148, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for HTTP/127.20.80.212@KRBTEST.COM
I20260929 10:52:29.431401 1649 heartbeater.cc:464] Registering TS with master...
I20260929 10:52:29.431581 1649 heartbeater.cc:511] Master 127.20.80.254:34443 requested a full tablet report, sending...
I20260929 10:52:29.431919 1180 ts_manager.cc:194] Registered new tserver with Master: 68a8f7cfde144e13a79cac6ba73aaecc (127.20.80.195:33103)
I20260929 10:52:29.432333 1180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.195@KRBTEST.COM'} at 127.20.80.195:52693
I20260929 10:52:29.435216 20803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:52:29.443043 1671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:29.436145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57900 (local address 127.20.80.254:34443)
0929 10:52:29.436253 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:29.436256 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:29.436310 (+ 54us) server_negotiation.cc:407] Connection header received
0929 10:52:29.436381 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:29.436382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:29.436394 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:29.436458 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:29.437066 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:29.437569 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:29.438314 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:29.438538 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:29.438620 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:29.439908 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:29.439910 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:29.439910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:29.439918 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:29.441783 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:29.442196 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:29.442197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:29.442198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:29.442261 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:29.442483 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:29.442484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:29.442484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:29.442597 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:29.442706 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:29.442803 (+ 97us) server_negotiation.cc:299] Negotiation successful
0929 10:52:29.442909 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":59,"threads_started":1}
I20260929 10:52:29.445065 1180 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57900:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:52:29.446827 1180 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:52:29.456561 1678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:29.449636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:37081 (local address 127.0.0.1:44288)
0929 10:52:29.450026 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:29.450069 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:29.450137 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:29.453275 (+ 3138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:29.453278 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:29.453281 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:52:29.453363 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:29.453365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:29.454237 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:29.454238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:29.455468 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:29.455471 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:29.456182 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:29.456184 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:29.456300 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:29.456318 (+ 18us) client_negotiation.cc:769] Sending connection context
0929 10:52:29.456418 (+ 100us) client_negotiation.cc:240] Negotiation successful
0929 10:52:29.456474 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":85,"threads_started":1}
I20260929 10:52:29.456563 1680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:29.449922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44288 (local address 127.20.80.194:37081)
0929 10:52:29.453096 (+ 3174us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:29.453102 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:29.453114 (+ 12us) server_negotiation.cc:407] Connection header received
0929 10:52:29.453151 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:29.453154 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:29.453171 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:29.453248 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:52:29.453456 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:29.454140 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:29.455559 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:29.456072 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:29.456178 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:29.456261 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:29.456438 (+ 177us) server_negotiation.cc:299] Negotiation successful
0929 10:52:29.456480 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3134,"thread_start_us":67,"threads_started":1}
I20260929 10:52:29.457669 1448 tablet_service.cc:1467] Processing CreateTablet for tablet 7e412510f15c4c7d9c4c51adf263b185 (DEFAULT_TABLE table=test-table [id=d73b6debd7364f9087044ecfbe85dc12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:52:29.458055 1448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e412510f15c4c7d9c4c51adf263b185. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:52:29.460844 1676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:29.449676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:33731 (local address 127.0.0.1:54812)
0929 10:52:29.453369 (+ 3693us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:29.453380 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:29.453424 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:29.457258 (+ 3834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:29.457259 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:29.457262 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:52:29.457303 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:29.457304 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:29.458542 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:29.458543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:29.459586 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:29.459589 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:29.460608 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:29.460609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:29.460670 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:29.460674 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:52:29.460725 (+ 51us) client_negotiation.cc:240] Negotiation successful
0929 10:52:29.460769 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3672,"mutex_wait_us":1,"thread_start_us":66,"threads_started":1}
I20260929 10:52:29.461130 1679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:29.449922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54812 (local address 127.20.80.193:33731)
0929 10:52:29.457082 (+ 7160us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:29.457087 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:29.457100 (+ 13us) server_negotiation.cc:407] Connection header received
0929 10:52:29.457138 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:29.457140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:29.457158 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:29.457252 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:52:29.457390 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:29.458442 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:29.459686 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:29.460430 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:29.460511 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:29.460570 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:29.460966 (+ 396us) server_negotiation.cc:299] Negotiation successful
0929 10:52:29.461036 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7118,"thread_start_us":65,"threads_started":1}
I20260929 10:52:29.461833 1313 tablet_service.cc:1467] Processing CreateTablet for tablet 7e412510f15c4c7d9c4c51adf263b185 (DEFAULT_TABLE table=test-table [id=d73b6debd7364f9087044ecfbe85dc12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:52:29.462199 1313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e412510f15c4c7d9c4c51adf263b185. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:52:29.462312 1682 tablet_bootstrap.cc:492] T 7e412510f15c4c7d9c4c51adf263b185 P 2189787f754d44b59de47ecebf6ab8ea: Bootstrap starting.
I20260929 10:52:29.462905 1682 tablet_bootstrap.cc:654] T 7e412510f15c4c7d9c4c51adf263b185 P 2189787f754d44b59de47ecebf6ab8ea: Neither blocks nor log segments found. Creating new log.
I20260929 10:52:29.463223 1682 log.cc:824] T 7e412510f15c4c7d9c4c51adf263b185 P 2189787f754d44b59de47ecebf6ab8ea: Log is configured to *not* fsync() on all Append() calls
I20260929 10:52:29.463950 1682 tablet_bootstrap.cc:492] T 7e412510f15c4c7d9c4c51adf263b185 P 2189787f754d44b59de47ecebf6ab8ea: No bootstrap required, opened a new log
I20260929 10:52:29.464038 1682 ts_tablet_manager.cc:1397] T 7e412510f15c4c7d9c4c51adf263b185 P 2189787f754d44b59de47ecebf6ab8ea: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:52:29.465669 1684 tablet_bootstrap.cc:492] T 7e412510f15c4c7d9c4c51adf263b185 P 3b7f4f8e0bb84d31a6119ec0e9085f78: Bootstrap starting.
I20260929 10:52:29.466373 1684 tablet_bootstrap.cc:654] T 7e412510f15c4c7d9c4c51adf263b185 P 3b7f4f8e0bb84d31a6119ec0e9085f78: Neither blocks nor log segments found. Creating new log.
I20260929 10:52:29.466467 1682 raft_consensus.cc:348] T 7e412510f15c4c7d9c4c51adf263b185 P 2189787f754d44b59de47ecebf6ab8ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2189787f754d44b59de47ecebf6ab8ea" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 37081 } } peers { permanent_uuid: "3b7f4f8e0bb84d31a6119ec0e9085f78" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 33731 } } peers { permanent_uuid: "68a8f7cfde144e13a79cac6ba73aaecc" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 33103 } }
I20260929 10:52:29.466730 1682 raft_consensus.cc:374] T 7e412510f15c4c7d9c4c51adf263b185 P 2189787f754d44b59de47ecebf6ab8ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:52:29.466761 1684 log.cc:824] T 7e412510f15c4c7d9c4c51adf263b185 P 3b7f4f8e0bb84d31a6119ec0e9085f78: Log is configured to *not* fsync() on all Append() calls
I20260929 10:52:29.466820 1682 raft_consensus.cc:729] T 7e412510f15c4c7d9c4c51adf263b185 P 2189787f754d44b59de47ecebf6ab8ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2189787f754d44b59de47ecebf6ab8ea, State: Initialized, Role: FOLLOWER
I20260929 10:52:29.467039 1682 consensus_queue.cc:260] T 7e412510f15c4c7d9c4c51adf263b185 P 2189787f754d44b59de47ecebf6ab8ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2189787f754d44b59de47ecebf6ab8ea" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 37081 } } peers { permanent_uuid: "3b7f4f8e0bb84d31a6119ec0e9085f78" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 33731 } } peers { permanent_uuid: "68a8f7cfde144e13a79cac6ba73aaecc" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 33103 } }
I20260929 10:52:29.467464 1682 ts_tablet_manager.cc:1428] T 7e412510f15c4c7d9c4c51adf263b185 P 2189787f754d44b59de47ecebf6ab8ea: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 10:52:29.467545 1684 tablet_bootstrap.cc:492] T 7e412510f15c4c7d9c4c51adf263b185 P 3b7f4f8e0bb84d31a6119ec0e9085f78: No bootstrap required, opened a new log
I20260929 10:52:29.467607 1684 ts_tablet_manager.cc:1397] T 7e412510f15c4c7d9c4c51adf263b185 P 3b7f4f8e0bb84d31a6119ec0e9085f78: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:52:29.467685 1514 heartbeater.cc:503] Master 127.20.80.254:34443 was elected leader, sending a full tablet report...
I20260929 10:52:29.469918 1678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:29.449873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:33103 (local address 127.0.0.1:48890)
0929 10:52:29.456955 (+ 7082us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:29.456966 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:29.457026 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:29.457708 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:29.457709 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:29.457711 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:52:29.457763 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:29.457764 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:29.459068 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:29.459069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:29.460148 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:29.460159 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:29.469625 (+ 9466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:29.469627 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:29.469706 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:29.469711 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 10:52:29.469777 (+ 66us) client_negotiation.cc:240] Negotiation successful
0929 10:52:29.469829 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7069,"thread_start_us":62,"threads_started":1}
I20260929 10:52:29.469975 1684 raft_consensus.cc:348] T 7e412510f15c4c7d9c4c51adf263b185 P 3b7f4f8e0bb84d31a6119ec0e9085f78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2189787f754d44b59de47ecebf6ab8ea" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 37081 } } peers { permanent_uuid: "3b7f4f8e0bb84d31a6119ec0e9085f78" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 33731 } } peers { permanent_uuid: "68a8f7cfde144e13a79cac6ba73aaecc" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 33103 } }
I20260929 10:52:29.470121 1684 raft_consensus.cc:374] T 7e412510f15c4c7d9c4c51adf263b185 P 3b7f4f8e0bb84d31a6119ec0e9085f78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:52:29.470162 1684 raft_consensus.cc:729] T 7e412510f15c4c7d9c4c51adf263b185 P 3b7f4f8e0bb84d31a6119ec0e9085f78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b7f4f8e0bb84d31a6119ec0e9085f78, State: Initialized, Role: FOLLOWER
I20260929 10:52:29.470211 1681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:29.450060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48890 (local address 127.20.80.195:33103)
0929 10:52:29.457456 (+ 7396us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:29.457460 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:29.457470 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:52:29.457503 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:29.457505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:29.457521 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:29.457600 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:52:29.457873 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:29.458950 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:29.461039 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:29.469508 (+ 8469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:29.469609 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:29.469675 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:29.470077 (+ 402us) server_negotiation.cc:299] Negotiation successful
0929 10:52:29.470126 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7359,"thread_start_us":69,"threads_started":1}
I20260929 10:52:29.470268 1684 consensus_queue.cc:260] T 7e412510f15c4c7d9c4c51adf263b185 P 3b7f4f8e0bb84d31a6119ec0e9085f78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2189787f754d44b59de47ecebf6ab8ea" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 37081 } } peers { permanent_uuid: "3b7f4f8e0bb84d31a6119ec0e9085f78" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 33731 } } peers { permanent_uuid: "68a8f7cfde144e13a79cac6ba73aaecc" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 33103 } }
I20260929 10:52:29.470551 1379 heartbeater.cc:503] Master 127.20.80.254:34443 was elected leader, sending a full tablet report...
I20260929 10:52:29.471289 1684 ts_tablet_manager.cc:1428] T 7e412510f15c4c7d9c4c51adf263b185 P 3b7f4f8e0bb84d31a6119ec0e9085f78: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260929 10:52:29.471365 1583 tablet_service.cc:1467] Processing CreateTablet for tablet 7e412510f15c4c7d9c4c51adf263b185 (DEFAULT_TABLE table=test-table [id=d73b6debd7364f9087044ecfbe85dc12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:52:29.471737 1583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e412510f15c4c7d9c4c51adf263b185. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:52:29.475545 1688 tablet_bootstrap.cc:492] T 7e412510f15c4c7d9c4c51adf263b185 P 68a8f7cfde144e13a79cac6ba73aaecc: Bootstrap starting.
I20260929 10:52:29.476243 1688 tablet_bootstrap.cc:654] T 7e412510f15c4c7d9c4c51adf263b185 P 68a8f7cfde144e13a79cac6ba73aaecc: Neither blocks nor log segments found. Creating new log.
I20260929 10:52:29.476588 1688 log.cc:824] T 7e412510f15c4c7d9c4c51adf263b185 P 68a8f7cfde144e13a79cac6ba73aaecc: Log is configured to *not* fsync() on all Append() calls
I20260929 10:52:29.477444 1688 tablet_bootstrap.cc:492] T 7e412510f15c4c7d9c4c51adf263b185 P 68a8f7cfde144e13a79cac6ba73aaecc: No bootstrap required, opened a new log
I20260929 10:52:29.477512 1688 ts_tablet_manager.cc:1397] T 7e412510f15c4c7d9c4c51adf263b185 P 68a8f7cfde144e13a79cac6ba73aaecc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:52:29.479940 1688 raft_consensus.cc:348] T 7e412510f15c4c7d9c4c51adf263b185 P 68a8f7cfde144e13a79cac6ba73aaecc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2189787f754d44b59de47ecebf6ab8ea" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 37081 } } peers { permanent_uuid: "3b7f4f8e0bb84d31a6119ec0e9085f78" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 33731 } } peers { permanent_uuid: "68a8f7cfde144e13a79cac6ba73aaecc" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 33103 } }
I20260929 10:52:29.480098 1688 raft_consensus.cc:374] T 7e412510f15c4c7d9c4c51adf263b185 P 68a8f7cfde144e13a79cac6ba73aaecc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:52:29.480127 1688 raft_consensus.cc:729] T 7e412510f15c4c7d9c4c51adf263b185 P 68a8f7cfde144e13a79cac6ba73aaecc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68a8f7cfde144e13a79cac6ba73aaecc, State: Initialized, Role: FOLLOWER
I20260929 10:52:29.480229 1688 consensus_queue.cc:260] T 7e412510f15c4c7d9c4c51adf263b185 P 68a8f7cfde144e13a79cac6ba73aaecc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2189787f754d44b59de47ecebf6ab8ea" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 37081 } } peers { permanent_uuid: "3b7f4f8e0bb84d31a6119ec0e9085f78" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 33731 } } peers { permanent_uuid: "68a8f7cfde144e13a79cac6ba73aaecc" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 33103 } }
I20260929 10:52:29.480511 1649 heartbeater.cc:503] Master 127.20.80.254:34443 was elected leader, sending a full tablet report...
I20260929 10:52:29.480811 1688 ts_tablet_manager.cc:1428] T 7e412510f15c4c7d9c4c51adf263b185 P 68a8f7cfde144e13a79cac6ba73aaecc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260929 10:52:29.488366 1515 tablet.cc:2369] T 7e412510f15c4c7d9c4c51adf263b185 P 2189787f754d44b59de47ecebf6ab8ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:52:29.545898 1380 tablet.cc:2369] T 7e412510f15c4c7d9c4c51adf263b185 P 3b7f4f8e0bb84d31a6119ec0e9085f78: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 29 10:52:29 dist-test-slave-vqch krb5kdc[32643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679148, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for HTTP/127.20.80.212@KRBTEST.COM
W20260929 10:52:29.666869 1650 tablet.cc:2369] T 7e412510f15c4c7d9c4c51adf263b185 P 68a8f7cfde144e13a79cac6ba73aaecc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:52:29.685741 1686 raft_consensus.cc:482] T 7e412510f15c4c7d9c4c51adf263b185 P 2189787f754d44b59de47ecebf6ab8ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:52:29.685911 1686 raft_consensus.cc:504] T 7e412510f15c4c7d9c4c51adf263b185 P 2189787f754d44b59de47ecebf6ab8ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2189787f754d44b59de47ecebf6ab8ea" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 37081 } } peers { permanent_uuid: "3b7f4f8e0bb84d31a6119ec0e9085f78" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 33731 } } peers { permanent_uuid: "68a8f7cfde144e13a79cac6ba73aaecc" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 33103 } }
I20260929 10:52:29.686326 1686 leader_election.cc:288] T 7e412510f15c4c7d9c4c51adf263b185 P 2189787f754d44b59de47ecebf6ab8ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b7f4f8e0bb84d31a6119ec0e9085f78 (127.20.80.193:33731), 68a8f7cfde144e13a79cac6ba73aaecc (127.20.80.195:33103)
I20260929 10:52:29.690606 1679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:29.686818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:33501 (local address 127.20.80.193:33731)
0929 10:52:29.686901 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:29.686904 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:29.687025 (+ 121us) server_negotiation.cc:407] Connection header received
0929 10:52:29.687133 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:29.687134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:29.687143 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:29.687174 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:52:29.687446 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:29.688158 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:29.689626 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:29.690195 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:29.690248 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:29.690258 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:29.690493 (+ 235us) server_negotiation.cc:299] Negotiation successful
0929 10:52:29.690536 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 10:52:29.690665 1681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:29.686799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:35751 (local address 127.20.80.195:33103)
0929 10:52:29.686845 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:29.686860 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:29.687032 (+ 172us) server_negotiation.cc:407] Connection header received
0929 10:52:29.687133 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:29.687134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:29.687143 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:29.687172 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:52:29.687403 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:29.688109 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:29.689710 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:29.690259 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:29.690315 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:29.690325 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:29.690549 (+ 224us) server_negotiation.cc:299] Negotiation successful
0929 10:52:29.690594 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 10:52:29.690925 1694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:29.686592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:33103 (local address 127.20.80.194:35751)
0929 10:52:29.686968 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:29.686979 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:29.687059 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:29.687237 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:29.687238 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:29.687240 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:52:29.687322 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:29.687324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:29.688213 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:29.688214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:29.689605 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:29.689608 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:29.690387 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:29.690388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:29.690467 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:29.690471 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:52:29.690777 (+ 306us) client_negotiation.cc:240] Negotiation successful
0929 10:52:29.690827 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":21,"thread_start_us":103,"threads_started":1}
I20260929 10:52:29.691324 1603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e412510f15c4c7d9c4c51adf263b185" candidate_uuid: "2189787f754d44b59de47ecebf6ab8ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68a8f7cfde144e13a79cac6ba73aaecc" is_pre_election: true
I20260929 10:52:29.691498 1603 raft_consensus.cc:2454] T 7e412510f15c4c7d9c4c51adf263b185 P 68a8f7cfde144e13a79cac6ba73aaecc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2189787f754d44b59de47ecebf6ab8ea in term 0.
I20260929 10:52:29.691623 1693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:29.686589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:33731 (local address 127.20.80.194:33501)
0929 10:52:29.686953 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:29.686966 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:29.687055 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:29.687243 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:29.687244 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:29.687246 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:52:29.687342 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:29.687344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:29.688262 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:29.688263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:29.689521 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:29.689524 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:29.690339 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:29.690340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:29.690406 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:29.690411 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 10:52:29.691063 (+ 652us) client_negotiation.cc:240] Negotiation successful
0929 10:52:29.691110 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":103,"threads_started":1}
I20260929 10:52:29.691746 1401 leader_election.cc:302] T 7e412510f15c4c7d9c4c51adf263b185 P 2189787f754d44b59de47ecebf6ab8ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2189787f754d44b59de47ecebf6ab8ea, 68a8f7cfde144e13a79cac6ba73aaecc; no voters:
I20260929 10:52:29.691919 1333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e412510f15c4c7d9c4c51adf263b185" candidate_uuid: "2189787f754d44b59de47ecebf6ab8ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7f4f8e0bb84d31a6119ec0e9085f78" is_pre_election: true
I20260929 10:52:29.692076 1333 raft_consensus.cc:2454] T 7e412510f15c4c7d9c4c51adf263b185 P 3b7f4f8e0bb84d31a6119ec0e9085f78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2189787f754d44b59de47ecebf6ab8ea in term 0.
I20260929 10:52:29.692180 1686 raft_consensus.cc:2781] T 7e412510f15c4c7d9c4c51adf263b185 P 2189787f754d44b59de47ecebf6ab8ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:52:29.692236 1686 raft_consensus.cc:482] T 7e412510f15c4c7d9c4c51adf263b185 P 2189787f754d44b59de47ecebf6ab8ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:52:29.692258 1686 raft_consensus.cc:3037] T 7e412510f15c4c7d9c4c51adf263b185 P 2189787f754d44b59de47ecebf6ab8ea [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:52:29.693228 1686 raft_consensus.cc:504] T 7e412510f15c4c7d9c4c51adf263b185 P 2189787f754d44b59de47ecebf6ab8ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2189787f754d44b59de47ecebf6ab8ea" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 37081 } } peers { permanent_uuid: "3b7f4f8e0bb84d31a6119ec0e9085f78" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 33731 } } peers { permanent_uuid: "68a8f7cfde144e13a79cac6ba73aaecc" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 33103 } }
I20260929 10:52:29.693365 1686 leader_election.cc:288] T 7e412510f15c4c7d9c4c51adf263b185 P 2189787f754d44b59de47ecebf6ab8ea [CANDIDATE]: Term 1 election: Requested vote from peers 3b7f4f8e0bb84d31a6119ec0e9085f78 (127.20.80.193:33731), 68a8f7cfde144e13a79cac6ba73aaecc (127.20.80.195:33103)
I20260929 10:52:29.694180 1333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e412510f15c4c7d9c4c51adf263b185" candidate_uuid: "2189787f754d44b59de47ecebf6ab8ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7f4f8e0bb84d31a6119ec0e9085f78"
I20260929 10:52:29.694265 1333 raft_consensus.cc:3037] T 7e412510f15c4c7d9c4c51adf263b185 P 3b7f4f8e0bb84d31a6119ec0e9085f78 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:52:29.695223 1333 raft_consensus.cc:2454] T 7e412510f15c4c7d9c4c51adf263b185 P 3b7f4f8e0bb84d31a6119ec0e9085f78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2189787f754d44b59de47ecebf6ab8ea in term 1.
I20260929 10:52:29.695423 1403 leader_election.cc:302] T 7e412510f15c4c7d9c4c51adf263b185 P 2189787f754d44b59de47ecebf6ab8ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2189787f754d44b59de47ecebf6ab8ea, 3b7f4f8e0bb84d31a6119ec0e9085f78; no voters:
I20260929 10:52:29.695605 1603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e412510f15c4c7d9c4c51adf263b185" candidate_uuid: "2189787f754d44b59de47ecebf6ab8ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68a8f7cfde144e13a79cac6ba73aaecc"
I20260929 10:52:29.695689 1603 raft_consensus.cc:3037] T 7e412510f15c4c7d9c4c51adf263b185 P 68a8f7cfde144e13a79cac6ba73aaecc [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:52:29.696554 1603 raft_consensus.cc:2454] T 7e412510f15c4c7d9c4c51adf263b185 P 68a8f7cfde144e13a79cac6ba73aaecc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2189787f754d44b59de47ecebf6ab8ea in term 1.
I20260929 10:52:29.696833 1686 raft_consensus.cc:2781] T 7e412510f15c4c7d9c4c51adf263b185 P 2189787f754d44b59de47ecebf6ab8ea [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:52:29.697012 1686 raft_consensus.cc:686] T 7e412510f15c4c7d9c4c51adf263b185 P 2189787f754d44b59de47ecebf6ab8ea [term 1 LEADER]: Becoming Leader. State: Replica: 2189787f754d44b59de47ecebf6ab8ea, State: Running, Role: LEADER
I20260929 10:52:29.697108 1686 consensus_queue.cc:237] T 7e412510f15c4c7d9c4c51adf263b185 P 2189787f754d44b59de47ecebf6ab8ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2189787f754d44b59de47ecebf6ab8ea" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 37081 } } peers { permanent_uuid: "3b7f4f8e0bb84d31a6119ec0e9085f78" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 33731 } } peers { permanent_uuid: "68a8f7cfde144e13a79cac6ba73aaecc" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 33103 } }
I20260929 10:52:29.697968 1180 catalog_manager.cc:5495] T 7e412510f15c4c7d9c4c51adf263b185 P 2189787f754d44b59de47ecebf6ab8ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 2189787f754d44b59de47ecebf6ab8ea (127.20.80.194). New cstate: current_term: 1 leader_uuid: "2189787f754d44b59de47ecebf6ab8ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2189787f754d44b59de47ecebf6ab8ea" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 37081 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b7f4f8e0bb84d31a6119ec0e9085f78" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 33731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68a8f7cfde144e13a79cac6ba73aaecc" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 33103 } health_report { overall_health: UNKNOWN } } }
I20260929 10:52:29.740038 1680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:29.736524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44304 (local address 127.20.80.194:37081)
0929 10:52:29.736567 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:29.736569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:29.736576 (+ 7us) server_negotiation.cc:407] Connection header received
0929 10:52:29.736616 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:29.736616 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:29.736638 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:29.736702 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:52:29.736899 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:29.737652 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:29.738861 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:29.739167 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:29.739244 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:29.739464 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:52:29.739603 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:52:29.739677 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:29.739931 (+ 254us) server_negotiation.cc:299] Negotiation successful
0929 10:52:29.739956 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 10:52:29.742317 1603 raft_consensus.cc:1261] T 7e412510f15c4c7d9c4c51adf263b185 P 68a8f7cfde144e13a79cac6ba73aaecc [term 1 FOLLOWER]: Refusing update from remote peer 2189787f754d44b59de47ecebf6ab8ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:52:29.742719 1686 consensus_queue.cc:1035] T 7e412510f15c4c7d9c4c51adf263b185 P 2189787f754d44b59de47ecebf6ab8ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "68a8f7cfde144e13a79cac6ba73aaecc" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 33103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:52:29.744897 1333 raft_consensus.cc:1261] T 7e412510f15c4c7d9c4c51adf263b185 P 3b7f4f8e0bb84d31a6119ec0e9085f78 [term 1 FOLLOWER]: Refusing update from remote peer 2189787f754d44b59de47ecebf6ab8ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:52:29.745241 1686 consensus_queue.cc:1035] T 7e412510f15c4c7d9c4c51adf263b185 P 2189787f754d44b59de47ecebf6ab8ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b7f4f8e0bb84d31a6119ec0e9085f78" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 33731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:52:29.748718 1698 mvcc.cc:205] Tried to move back new op lower bound from 7334621797341696000 to 7334621797159829504. Current Snapshot: MvccSnapshot[applied={T|T < 7334621797341696000}]
I20260929 10:52:29.750197 1699 mvcc.cc:205] Tried to move back new op lower bound from 7334621797341696000 to 7334621797159829504. Current Snapshot: MvccSnapshot[applied={T|T < 7334621797341696000}]
I20260929 10:52:29.751323 1179 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:52:29.751427 1179 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:52:29.752027 1701 mvcc.cc:205] Tried to move back new op lower bound from 7334621797341696000 to 7334621797159829504. Current Snapshot: MvccSnapshot[applied={T|T < 7334621797341696000}]
I20260929 10:52:29.752374 1179 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cb6095ecfe6241c8b8f5d23cf59acbd5: Sending DeleteTablet for 3 replicas of tablet 7e412510f15c4c7d9c4c51adf263b185
I20260929 10:52:29.752758 1313 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e412510f15c4c7d9c4c51adf263b185 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:52:29 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:54812
I20260929 10:52:29.753481 1448 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e412510f15c4c7d9c4c51adf263b185 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:52:29 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:44288
I20260929 10:52:29.753897 1583 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e412510f15c4c7d9c4c51adf263b185 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:52:29 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:48890
I20260929 10:52:29.754484 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 1218
I20260929 10:52:29.757267 1710 tablet_replica.cc:330] T 7e412510f15c4c7d9c4c51adf263b185 P 2189787f754d44b59de47ecebf6ab8ea: stopping tablet replica
I20260929 10:52:29.757368 1710 raft_consensus.cc:2229] T 7e412510f15c4c7d9c4c51adf263b185 P 2189787f754d44b59de47ecebf6ab8ea [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:52:29.757464 1710 raft_consensus.cc:2258] T 7e412510f15c4c7d9c4c51adf263b185 P 2189787f754d44b59de47ecebf6ab8ea [term 1 FOLLOWER]: Raft consensus is shut down!
W20260929 10:52:29.761412 1166 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.80.193:33731 (error 108)
I20260929 10:52:29.762313 1678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:29.762200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:33731 (local address 127.0.0.1:54824)
0929 10:52:29.762246 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:29.762281 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.193:33731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260929 10:52:29.762509 1166 catalog_manager.cc:4584] TS 3b7f4f8e0bb84d31a6119ec0e9085f78 (127.20.80.193:33731): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7e412510f15c4c7d9c4c51adf263b185: Network error: Client connection negotiation failed: client connection to 127.20.80.193:33731: connect: Connection refused (error 111)
I20260929 10:52:29.762889 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 1383
I20260929 10:52:29.763448 1711 tablet_replica.cc:330] T 7e412510f15c4c7d9c4c51adf263b185 P 68a8f7cfde144e13a79cac6ba73aaecc: stopping tablet replica
I20260929 10:52:29.763561 1711 raft_consensus.cc:2229] T 7e412510f15c4c7d9c4c51adf263b185 P 68a8f7cfde144e13a79cac6ba73aaecc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:52:29.763664 1711 raft_consensus.cc:2258] T 7e412510f15c4c7d9c4c51adf263b185 P 68a8f7cfde144e13a79cac6ba73aaecc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:52:29.764223 1711 ts_tablet_manager.cc:1905] T 7e412510f15c4c7d9c4c51adf263b185 P 68a8f7cfde144e13a79cac6ba73aaecc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:52:29.766197 1711 ts_tablet_manager.cc:1918] T 7e412510f15c4c7d9c4c51adf263b185 P 68a8f7cfde144e13a79cac6ba73aaecc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:52:29.766258 1711 log.cc:1196] T 7e412510f15c4c7d9c4c51adf263b185 P 68a8f7cfde144e13a79cac6ba73aaecc: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790678996126930-20803-0/minicluster-data/ts-2/wal/wals/7e412510f15c4c7d9c4c51adf263b185
I20260929 10:52:29.766508 1711 ts_tablet_manager.cc:1939] T 7e412510f15c4c7d9c4c51adf263b185 P 68a8f7cfde144e13a79cac6ba73aaecc: Deleting consensus metadata
I20260929 10:52:29.769037 1164 catalog_manager.cc:4841] TS 68a8f7cfde144e13a79cac6ba73aaecc (127.20.80.195:33103): tablet 7e412510f15c4c7d9c4c51adf263b185 (table test-table [id=d73b6debd7364f9087044ecfbe85dc12]) successfully deleted
I20260929 10:52:29.771294 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 1516
I20260929 10:52:29.772390 1678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:29.771369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:37081 (local address 127.0.0.1:44316)
0929 10:52:29.772315 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:29.772353 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.194:37081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":924}
I20260929 10:52:29.778715 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 1147
I20260929 10:52:29.786679 20803 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260929 10:52:30.131217 20803 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260929 10:52:30.131332 20803 mini_ranger.cc:66] Stopping Ranger...
I20260929 10:52:30.475566 20803 mini_ranger.cc:68] Stopped Ranger
2026-09-29 10:52:30.475 UTC [32674] LOG: received smart shutdown request
2026-09-29 10:52:30.479 UTC [32674] LOG: background worker "logical replication launcher" (PID 32683) exited with exit code 1
2026-09-29 10:52:30.479 UTC [32678] LOG: shutting down
2026-09-29 10:52:30.690 UTC [32674] LOG: database system is shut down
2026-09-29T10:52:30Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49066 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:52:30 dist-test-slave-vqch krb5kdc[1721](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:52:30 dist-test-slave-vqch krb5kdc[1721](info): set up 2 sockets
Sep 29 10:52:30 dist-test-slave-vqch krb5kdc[1721](info): commencing operation
krb5kdc: starting...
W20260929 10:52:31.844682 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:52:31 dist-test-slave-vqch krb5kdc[1721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:52:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:52:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.80.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
I20260929 10:52:31.968840 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:46833
--webserver_interface=127.20.80.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:41699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:46833,127.20.80.253:34087,127.20.80.252:37433
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:52:32.049458 1734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:52:32.049714 1734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:52:32.049752 1734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:52:32.050917 1734 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:52:32.050992 1734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:52:32.051016 1734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:52:32.051043 1734 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:52:32.051072 1734 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:52:32.052587 1734 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:41699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:46833,127.20.80.253:34087,127.20.80.252:37433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:46833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:52:32.052862 1734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:52:32.053119 1734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:52:32.058142 1742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:32.058238 1734 server_base.cc:1034] running on GCE node
W20260929 10:52:32.058151 1740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:32.058184 1739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:32.058583 1734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:52:32.058838 1734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:52:32.059980 1734 hybrid_clock.cc:648] HybridClock initialized: now 1790679152059966 us; error 33 us; skew 500 ppm
Sep 29 10:52:32 dist-test-slave-vqch krb5kdc[1721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679152, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:52:32.066148 1734 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:52:32.066636 1734 webserver.cc:460] Webserver started at http://127.20.80.254:39385/ using document root <none> and password file <none>
I20260929 10:52:32.066826 1734 fs_manager.cc:359] Metadata directory not provided
I20260929 10:52:32.066867 1734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:52:32.066968 1734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:52:32.068001 1734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "8860fcd62a3643409b4308b54568bbb6"
format_stamp: "Formatted at 2026-09-29 10:52:32 on dist-test-slave-vqch"
server_key: "ae7e02d0ec8711de0cb593b6d5176585"
server_key_iv: "0d532d3ac025e14c3450e258f2e04f01"
server_key_version: "encryptionkey@0"
I20260929 10:52:32.068305 1734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "8860fcd62a3643409b4308b54568bbb6"
format_stamp: "Formatted at 2026-09-29 10:52:32 on dist-test-slave-vqch"
server_key: "ae7e02d0ec8711de0cb593b6d5176585"
server_key_iv: "0d532d3ac025e14c3450e258f2e04f01"
server_key_version: "encryptionkey@0"
I20260929 10:52:32.069705 1734 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260929 10:52:32.070528 1749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:32.070780 1734 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:52:32.070873 1734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "8860fcd62a3643409b4308b54568bbb6"
format_stamp: "Formatted at 2026-09-29 10:52:32 on dist-test-slave-vqch"
server_key: "ae7e02d0ec8711de0cb593b6d5176585"
server_key_iv: "0d532d3ac025e14c3450e258f2e04f01"
server_key_version: "encryptionkey@0"
I20260929 10:52:32.070948 1734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:52:32.092365 1734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:52:32.094448 1734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:52:32.094564 1734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:52:32.098510 1734 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:46833
I20260929 10:52:32.098592 1803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:46833 every 8 connection(s)
I20260929 10:52:32.098938 1734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:52:32.099867 1804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:52:32.101519 1804 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46833 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:52:32.103749 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 1734
I20260929 10:52:32.103865 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:52:32.104058 20803 external_mini_cluster.cc:1428] Setting key 845428fac6ad3bf4269fb99cff3d4faf
Sep 29 10:52:32 dist-test-slave-vqch krb5kdc[1721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679152, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.80.253@KRBTEST.COM; defaulting to no policy
I20260929 10:52:32.112097 1807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.102277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44514 (local address 127.20.80.254:46833)
0929 10:52:32.102457 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:32.102467 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:32.102556 (+ 89us) server_negotiation.cc:407] Connection header received
0929 10:52:32.103027 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:32.103029 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:32.103045 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:32.103143 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:32.103703 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.105105 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.106345 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.106652 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.106734 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:32.108826 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:32.108834 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:32.108835 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:32.108841 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:32.110481 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:32.110886 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:32.110899 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:32.110899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:32.110983 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:32.111221 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:32.111221 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:32.111222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:32.111442 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:32.111529 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:32.111724 (+ 195us) server_negotiation.cc:299] Negotiation successful
0929 10:52:32.111847 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":118,"threads_started":1}
I20260929 10:52:32.112097 1806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.102241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:46833 (local address 127.0.0.1:44514)
0929 10:52:32.102415 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:32.102457 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:32.102906 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:32.103153 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:32.103155 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:32.103404 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:52:32.103588 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:32.103592 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.105193 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.105194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:32.106248 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:32.106251 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.106736 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.106737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:32.106798 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:32.107359 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:52:32.107367 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:52:32.108730 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:52:32.110587 (+ 1857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:32.110595 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:32.110597 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:32.110787 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:32.111053 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:32.111054 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:32.111054 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:32.111144 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:32.111524 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:52:32.111529 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:52:32.111590 (+ 61us) client_negotiation.cc:769] Sending connection context
0929 10:52:32.111699 (+ 109us) client_negotiation.cc:240] Negotiation successful
0929 10:52:32.111930 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":84,"thread_start_us":93,"threads_started":1}
I20260929 10:52:32.113060 1804 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.80.253" port: 34087 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:52:32.113570 1806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.113335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:34087 (local address 127.0.0.1:58018)
0929 10:52:32.113512 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:32.113550 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.253:34087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162}
W20260929 10:52:32.113698 1751 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.80.253:34087: connect: Connection refused (error 111)
I20260929 10:52:32.114078 1806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.113977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:34087 (local address 127.0.0.1:58028)
0929 10:52:32.114030 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:32.114055 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.253:34087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260929 10:52:32.114187 1804 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.80.253:34087: Network error: Client connection negotiation failed: client connection to 127.20.80.253:34087: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.253.keytab.
Entry for principal kudu/127.20.80.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.253.keytab.
I20260929 10:52:32.131532 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.253:34087
--webserver_interface=127.20.80.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.253.keytab
--principal=kudu/127.20.80.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:41699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:46833,127.20.80.253:34087,127.20.80.252:37433
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:52:32.163483 1804 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.80.253" port: 34087 } attempt: 1
I20260929 10:52:32.164052 1806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.163906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:34087 (local address 127.0.0.1:58042)
0929 10:52:32.163974 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:32.164019 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.253:34087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260929 10:52:32.164438 1806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.164358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:34087 (local address 127.0.0.1:58044)
0929 10:52:32.164397 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:32.164422 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.253:34087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260929 10:52:32.164564 1804 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.80.253:34087: Network error: Client connection negotiation failed: client connection to 127.20.80.253:34087: connect: Connection refused (error 111)
W20260929 10:52:32.212944 1811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:52:32.213227 1811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:52:32.213250 1811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:52:32.214380 1811 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:52:32.214421 1811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:52:32.214434 1811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:52:32.214447 1811 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:52:32.214502 1811 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:52:32.216061 1811 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:41699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:46833,127.20.80.253:34087,127.20.80.252:37433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.253.keytab
--principal=kudu/127.20.80.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.253:34087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.80.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:52:32.216312 1811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:52:32.216538 1811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:52:32.221342 1819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:32.221356 1817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:32.221603 1811 server_base.cc:1034] running on GCE node
W20260929 10:52:32.221341 1816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:32.221948 1811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:52:32.222266 1811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:52:32.223439 1811 hybrid_clock.cc:648] HybridClock initialized: now 1790679152223426 us; error 27 us; skew 500 ppm
Sep 29 10:52:32 dist-test-slave-vqch krb5kdc[1721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679152, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:52:32.226029 1811 init.cc:377] Logged in from keytab as kudu/127.20.80.253@KRBTEST.COM (short username kudu)
I20260929 10:52:32.226411 1811 webserver.cc:460] Webserver started at http://127.20.80.253:46065/ using document root <none> and password file <none>
I20260929 10:52:32.226650 1811 fs_manager.cc:359] Metadata directory not provided
I20260929 10:52:32.226720 1811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:52:32.226887 1811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:52:32.227947 1811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-1/data/instance:
uuid: "0327385e58754995b8e7021dd9821c5e"
format_stamp: "Formatted at 2026-09-29 10:52:32 on dist-test-slave-vqch"
server_key: "e7fe499a83947e0180eed4b3c34c3ccd"
server_key_iv: "7a4a5cc70d8f335ce6d671c01bdb14ce"
server_key_version: "encryptionkey@0"
I20260929 10:52:32.228320 1811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-1/wal/instance:
uuid: "0327385e58754995b8e7021dd9821c5e"
format_stamp: "Formatted at 2026-09-29 10:52:32 on dist-test-slave-vqch"
server_key: "e7fe499a83947e0180eed4b3c34c3ccd"
server_key_iv: "7a4a5cc70d8f335ce6d671c01bdb14ce"
server_key_version: "encryptionkey@0"
I20260929 10:52:32.229740 1811 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 10:52:32.230554 1826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:32.230733 1811 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 10:52:32.230834 1811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-1/wal
uuid: "0327385e58754995b8e7021dd9821c5e"
format_stamp: "Formatted at 2026-09-29 10:52:32 on dist-test-slave-vqch"
server_key: "e7fe499a83947e0180eed4b3c34c3ccd"
server_key_iv: "7a4a5cc70d8f335ce6d671c01bdb14ce"
server_key_version: "encryptionkey@0"
I20260929 10:52:32.230932 1811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:52:32.232746 1804 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.80.253" port: 34087 } attempt: 2
I20260929 10:52:32.233184 1806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.233061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:34087 (local address 127.0.0.1:58048)
0929 10:52:32.233117 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:32.233154 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.253:34087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260929 10:52:32.233520 1806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.233431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:34087 (local address 127.0.0.1:58056)
0929 10:52:32.233473 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:32.233496 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.253:34087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260929 10:52:32.233608 1804 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.80.253:34087: Network error: Client connection negotiation failed: client connection to 127.20.80.253:34087: connect: Connection refused (error 111)
I20260929 10:52:32.241307 1811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:52:32.243671 1811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:52:32.243785 1811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:52:32.247763 1811 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.253:34087
I20260929 10:52:32.247859 1879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.253:34087 every 8 connection(s)
I20260929 10:52:32.248168 1811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-1/data/info.pb
I20260929 10:52:32.249148 1880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:52:32.250659 1880 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46833 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:52:32.256384 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 1811
I20260929 10:52:32.256467 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-1/wal/instance
Sep 29 10:52:32 dist-test-slave-vqch krb5kdc[1721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679152, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.253@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:52:32.256742 20803 external_mini_cluster.cc:1428] Setting key cdd463b0a9be542baac4fe99e96616e7
I20260929 10:52:32.260352 1882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.251433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:46833 (local address 127.0.0.1:44522)
0929 10:52:32.251645 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:32.251688 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:32.252225 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:32.252510 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:32.252514 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:32.252758 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:52:32.252946 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:32.252949 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.253711 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.253712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:32.254603 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:32.254606 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.254996 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.254997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:32.255124 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:32.255567 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:52:32.255574 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:52:32.257092 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:52:32.258810 (+ 1718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:32.258818 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:32.258821 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:32.259165 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:32.259436 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:32.259437 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:32.259437 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:32.259552 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:32.259843 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:52:32.259847 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:52:32.259982 (+ 135us) client_negotiation.cc:769] Sending connection context
0929 10:52:32.260066 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 10:52:32.260192 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":93,"threads_started":1}
I20260929 10:52:32.260362 1807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.251452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44522 (local address 127.20.80.254:46833)
0929 10:52:32.251494 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:32.251496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:32.251787 (+ 291us) server_negotiation.cc:407] Connection header received
0929 10:52:32.252356 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:32.252358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:32.252372 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:32.252428 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:32.253093 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.253600 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.254696 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.254913 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.254991 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:32.257220 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:32.257222 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:32.257222 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:32.257231 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:32.258722 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:32.259252 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:32.259253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:32.259254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:32.259339 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:32.259620 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:32.259621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:32.259622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:32.259774 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:32.259847 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:32.260122 (+ 275us) server_negotiation.cc:299] Negotiation successful
0929 10:52:32.260228 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 10:52:32.260969 1880 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.80.253" port: 34087 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.20.80.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.252@KRBTEST.COM" created.
Sep 29 10:52:32 dist-test-slave-vqch krb5kdc[1721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679152, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.253@KRBTEST.COM for kudu/127.20.80.253@KRBTEST.COM
I20260929 10:52:32.271337 1884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.261462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58058 (local address 127.20.80.253:34087)
0929 10:52:32.261621 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:32.261624 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:32.261641 (+ 17us) server_negotiation.cc:407] Connection header received
0929 10:52:32.261659 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:32.261660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:32.261671 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:32.261813 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:32.262111 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.262816 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.263546 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.263870 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.263965 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:32.268180 (+ 4215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:32.268192 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:32.268192 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:32.268201 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:32.269991 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:32.270427 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:32.270429 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:32.270430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:32.270484 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:32.270685 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:32.270686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:32.270686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:32.270842 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:32.270906 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:32.271114 (+ 208us) server_negotiation.cc:299] Negotiation successful
0929 10:52:32.271242 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":133,"thread_start_us":79,"threads_started":1}
I20260929 10:52:32.271353 1882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.261301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:34087 (local address 127.0.0.1:58058)
0929 10:52:32.261383 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:32.261395 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:32.261447 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:32.261816 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:32.261816 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:32.261967 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:52:32.262021 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:32.262022 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.262927 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.262928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:32.263443 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:32.263445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.263964 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.263965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:32.264032 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:32.264556 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:52:32.264562 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:52:32.268083 (+ 3521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:52:32.270086 (+ 2003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:32.270087 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:32.270088 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:32.270328 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:32.270551 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:32.270552 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:32.270552 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:32.270623 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:32.270917 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:52:32.270918 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:52:32.270976 (+ 58us) client_negotiation.cc:769] Sending connection context
0929 10:52:32.271058 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 10:52:32.271183 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260929 10:52:32.272099 1880 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37433 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:52:32.272513 1882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.272357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:37433 (local address 127.0.0.1:35428)
0929 10:52:32.272424 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:32.272481 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.252:37433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260929 10:52:32.272699 1828 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.80.252:37433: connect: Connection refused (error 111)
I20260929 10:52:32.273300 1882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.273179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:37433 (local address 127.0.0.1:35430)
0929 10:52:32.273234 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:32.273263 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.252:37433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260929 10:52:32.273450 1880 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.80.252:37433: Network error: Client connection negotiation failed: client connection to 127.20.80.252:37433: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.252.keytab.
Entry for principal kudu/127.20.80.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.252.keytab.
I20260929 10:52:32.284825 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.252:37433
--webserver_interface=127.20.80.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.252.keytab
--principal=kudu/127.20.80.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:41699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:46833,127.20.80.253:34087,127.20.80.252:37433
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:52:32.322713 1880 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37433 } attempt: 1
I20260929 10:52:32.323274 1882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.323069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:37433 (local address 127.0.0.1:35436)
0929 10:52:32.323151 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:32.323223 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.252:37433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260929 10:52:32.323701 1882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.323581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:37433 (local address 127.0.0.1:35440)
0929 10:52:32.323639 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:32.323673 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.252:37433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260929 10:52:32.323839 1880 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.80.252:37433: Network error: Client connection negotiation failed: client connection to 127.20.80.252:37433: connect: Connection refused (error 111)
I20260929 10:52:32.324749 1804 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.80.253" port: 34087 } attempt: 3
Sep 29 10:52:32 dist-test-slave-vqch krb5kdc[1721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679152, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for kudu/127.20.80.253@KRBTEST.COM
I20260929 10:52:32.332069 1884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.325093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58072 (local address 127.20.80.253:34087)
0929 10:52:32.325143 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:32.325145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:32.325156 (+ 11us) server_negotiation.cc:407] Connection header received
0929 10:52:32.325208 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:32.325209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:32.325219 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:32.325264 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:32.325633 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.326122 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.327141 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.327354 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.327419 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:32.329496 (+ 2077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:32.329498 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:32.329498 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:32.329505 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:32.330852 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:32.331250 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:32.331251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:32.331252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:32.331311 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:32.331525 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:32.331526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:32.331527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:32.331644 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:32.331720 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:32.331897 (+ 177us) server_negotiation.cc:299] Negotiation successful
0929 10:52:32.331974 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260929 10:52:32.332070 1806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.325047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:34087 (local address 127.0.0.1:58072)
0929 10:52:32.325095 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:32.325106 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:32.325147 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:32.325290 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:32.325291 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:32.325482 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:52:32.325559 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:32.325561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.326211 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.326213 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:32.327050 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:32.327054 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.327431 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.327432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:32.327512 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:32.327944 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:52:32.327949 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:52:32.329405 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:52:32.330977 (+ 1572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:32.330978 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:32.330978 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:32.331151 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:32.331385 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:32.331386 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:32.331387 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:32.331458 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:32.331723 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:52:32.331724 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:52:32.331782 (+ 58us) client_negotiation.cc:769] Sending connection context
0929 10:52:32.331864 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 10:52:32.331974 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260929 10:52:32.332655 1804 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37433 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:52:32.333078 1806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.332954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:37433 (local address 127.0.0.1:35446)
0929 10:52:32.333020 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:32.333048 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.252:37433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260929 10:52:32.333447 1806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.333359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:37433 (local address 127.0.0.1:35462)
0929 10:52:32.333404 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:32.333429 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.252:37433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260929 10:52:32.333554 1804 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.80.252:37433: Network error: Client connection negotiation failed: client connection to 127.20.80.252:37433: connect: Connection refused (error 111)
I20260929 10:52:32.364712 1804 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37433 } attempt: 1
I20260929 10:52:32.365275 1806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.365136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:37433 (local address 127.0.0.1:35466)
0929 10:52:32.365199 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:32.365245 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.252:37433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260929 10:52:32.365720 1806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.365624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:37433 (local address 127.0.0.1:35480)
0929 10:52:32.365674 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:32.365700 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.252:37433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260929 10:52:32.365839 1804 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.80.252:37433: Network error: Client connection negotiation failed: client connection to 127.20.80.252:37433: connect: Connection refused (error 111)
W20260929 10:52:32.370803 1887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:52:32.371026 1887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:52:32.371074 1887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:52:32.372179 1887 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:52:32.372242 1887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:52:32.372267 1887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:52:32.372294 1887 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:52:32.372323 1887 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:52:32.373979 1887 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:41699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:46833,127.20.80.253:34087,127.20.80.252:37433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.252.keytab
--principal=kudu/127.20.80.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.252:37433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.80.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:52:32.374224 1887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:52:32.374393 1887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:52:32.379076 1895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:32.379122 1893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:32.379084 1892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:32.379395 1887 server_base.cc:1034] running on GCE node
I20260929 10:52:32.379628 1887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:52:32.379863 1887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:52:32.381052 1887 hybrid_clock.cc:648] HybridClock initialized: now 1790679152381029 us; error 31 us; skew 500 ppm
Sep 29 10:52:32 dist-test-slave-vqch krb5kdc[1721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679152, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:52:32.383598 1887 init.cc:377] Logged in from keytab as kudu/127.20.80.252@KRBTEST.COM (short username kudu)
I20260929 10:52:32.384070 1887 webserver.cc:460] Webserver started at http://127.20.80.252:33897/ using document root <none> and password file <none>
I20260929 10:52:32.384301 1887 fs_manager.cc:359] Metadata directory not provided
I20260929 10:52:32.384362 1887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:52:32.384528 1887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:52:32.385622 1887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-2/data/instance:
uuid: "0135d6ff59d040f8b68301e68a5faebe"
format_stamp: "Formatted at 2026-09-29 10:52:32 on dist-test-slave-vqch"
server_key: "b09809807c7abd21422af48a98dae14c"
server_key_iv: "e0ea24660dc23d08d1c691abc3455f68"
server_key_version: "encryptionkey@0"
I20260929 10:52:32.385974 1887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-2/wal/instance:
uuid: "0135d6ff59d040f8b68301e68a5faebe"
format_stamp: "Formatted at 2026-09-29 10:52:32 on dist-test-slave-vqch"
server_key: "b09809807c7abd21422af48a98dae14c"
server_key_iv: "e0ea24660dc23d08d1c691abc3455f68"
server_key_version: "encryptionkey@0"
I20260929 10:52:32.387274 1887 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:52:32.388141 1902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:32.388341 1887 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:52:32.388411 1887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-2/wal
uuid: "0135d6ff59d040f8b68301e68a5faebe"
format_stamp: "Formatted at 2026-09-29 10:52:32 on dist-test-slave-vqch"
server_key: "b09809807c7abd21422af48a98dae14c"
server_key_iv: "e0ea24660dc23d08d1c691abc3455f68"
server_key_version: "encryptionkey@0"
I20260929 10:52:32.388497 1887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:52:32.392045 1880 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37433 } attempt: 2
I20260929 10:52:32.392504 1882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.392359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:37433 (local address 127.0.0.1:35496)
0929 10:52:32.392433 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:32.392470 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.252:37433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260929 10:52:32.392876 1882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.392790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:37433 (local address 127.0.0.1:35502)
0929 10:52:32.392836 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:32.392859 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.252:37433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260929 10:52:32.392968 1880 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.80.252:37433: Network error: Client connection negotiation failed: client connection to 127.20.80.252:37433: connect: Connection refused (error 111)
I20260929 10:52:32.398809 1887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:52:32.401132 1887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:52:32.401284 1887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:52:32.405172 1887 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.252:37433
I20260929 10:52:32.405222 1955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.252:37433 every 8 connection(s)
I20260929 10:52:32.405586 1887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-2/data/info.pb
I20260929 10:52:32.406416 1956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:52:32.407943 1956 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46833 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:52:32.409919 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 1887
I20260929 10:52:32.410020 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790678996126930-20803-0/minicluster-data/master-2/wal/instance
I20260929 10:52:32.410238 20803 external_mini_cluster.cc:1428] Setting key 9ab223aa5650970b6800dea0b2f0cb66
Sep 29 10:52:32 dist-test-slave-vqch krb5kdc[1721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679152, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.252@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
Sep 29 10:52:32 dist-test-slave-vqch krb5kdc[1721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.252@KRBTEST.COM
I20260929 10:52:32.419782 1807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.408671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44538 (local address 127.20.80.254:46833)
0929 10:52:32.408734 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:32.408736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:32.408905 (+ 169us) server_negotiation.cc:407] Connection header received
0929 10:52:32.409414 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:32.409415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:32.409424 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:32.409481 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:32.410301 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.411016 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.412193 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.412496 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.412572 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:32.415210 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:32.415212 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:32.415212 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:32.415220 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:32.418030 (+ 2810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:32.418592 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:32.418593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:32.418593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:32.418657 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:32.418978 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:32.418979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:32.418979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:32.419147 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:32.419226 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:32.419596 (+ 370us) server_negotiation.cc:299] Negotiation successful
0929 10:52:32.419681 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 10:52:32.420063 1958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.408627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:46833 (local address 127.0.0.1:44538)
0929 10:52:32.408779 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:32.408820 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:32.409326 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:32.409545 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:32.409549 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:32.409855 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:52:32.410182 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:32.410189 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.411104 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.411105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:32.412108 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:32.412110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.412592 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.412593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:32.412737 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:32.413271 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:52:32.413282 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:52:32.415114 (+ 1832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:52:32.418114 (+ 3000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:32.418121 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:32.418124 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:32.418497 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:32.418735 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:32.418736 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:32.418736 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:32.418878 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:32.419259 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:52:32.419267 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:52:32.419439 (+ 172us) client_negotiation.cc:769] Sending connection context
0929 10:52:32.419564 (+ 125us) client_negotiation.cc:240] Negotiation successful
0929 10:52:32.419733 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":77,"threads_started":1}
I20260929 10:52:32.420742 1956 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.80.253" port: 34087 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:52:32.423717 1960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.410610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35506 (local address 127.20.80.252:37433)
0929 10:52:32.410765 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:32.410769 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:32.410786 (+ 17us) server_negotiation.cc:407] Connection header received
0929 10:52:32.410832 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:32.410835 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:32.410855 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:32.410922 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:32.411649 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.412493 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.413466 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.413745 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.413828 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:32.417775 (+ 3947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:32.417784 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:32.417784 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:32.417791 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:32.422201 (+ 4410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:32.422613 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:32.422616 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:32.422617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:32.422675 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:32.422865 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:32.422866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:32.422866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:32.423033 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:32.423179 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:32.423546 (+ 367us) server_negotiation.cc:299] Negotiation successful
0929 10:52:32.423617 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":85,"threads_started":1}
Sep 29 10:52:32 dist-test-slave-vqch krb5kdc[1721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679152, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.252@KRBTEST.COM for kudu/127.20.80.253@KRBTEST.COM
Sep 29 10:52:32 dist-test-slave-vqch krb5kdc[1721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:52:32.428690 1958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.421084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:34087 (local address 127.0.0.1:58078)
0929 10:52:32.421130 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:32.421141 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:32.421182 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:32.421794 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:32.421795 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:32.422046 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:52:32.422091 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:32.422092 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.423040 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.423041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:32.423780 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:32.423783 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.424253 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.424254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:32.424326 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:32.424799 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:52:32.424804 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:52:32.426003 (+ 1199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:52:32.427714 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:32.427715 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:32.427716 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:32.427887 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:32.428092 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:32.428093 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:32.428093 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:32.428149 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:32.428421 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:52:32.428422 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:52:32.428463 (+ 41us) client_negotiation.cc:769] Sending connection context
0929 10:52:32.428553 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 10:52:32.428634 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260929 10:52:32.428774 1884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.421619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58078 (local address 127.20.80.253:34087)
0929 10:52:32.421666 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:32.421668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:32.421677 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:52:32.421701 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:32.421702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:32.421713 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:32.421773 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:32.422201 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.422936 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.423901 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.424172 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.424243 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:32.426096 (+ 1853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:32.426097 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:32.426098 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:32.426105 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:32.427612 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:32.427965 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:32.427966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:32.427966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:32.428023 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:32.428225 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:32.428226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:32.428226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:32.428346 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:32.428418 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:32.428589 (+ 171us) server_negotiation.cc:299] Negotiation successful
0929 10:52:32.428670 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 10:52:32.429153 1956 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37433 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:52:32.431764 1807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.424595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44554 (local address 127.20.80.254:46833)
0929 10:52:32.424634 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:32.424636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:32.424680 (+ 44us) server_negotiation.cc:407] Connection header received
0929 10:52:32.424754 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:32.424755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:32.424766 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:32.424820 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:32.425375 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.425847 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.426630 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.426832 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.426917 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:32.428939 (+ 2022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:32.428940 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:32.428940 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:32.428948 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:32.430662 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:32.431064 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:32.431064 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:32.431065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:32.431115 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:32.431315 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:32.431315 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:32.431315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:32.431425 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:32.431562 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:32.431623 (+ 61us) server_negotiation.cc:299] Negotiation successful
0929 10:52:32.431678 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Sep 29 10:52:32 dist-test-slave-vqch krb5kdc[1721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679152, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.252@KRBTEST.COM for kudu/127.20.80.252@KRBTEST.COM
I20260929 10:52:32.437120 1958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.429432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:37433 (local address 127.0.0.1:35512)
0929 10:52:32.429731 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:32.429741 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:32.429791 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:32.429950 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:32.429950 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:32.430067 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:52:32.430095 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:32.430096 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.430954 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.430955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:32.431700 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:32.431703 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.432239 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.432240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:32.432307 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:32.432758 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:52:32.432764 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:52:32.434216 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:52:32.435852 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:32.435852 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:32.435852 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:32.436184 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:32.436431 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:32.436432 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:32.436432 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:32.436486 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:32.436807 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:52:32.436808 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:52:32.436844 (+ 36us) client_negotiation.cc:769] Sending connection context
0929 10:52:32.436909 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 10:52:32.437060 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281}
I20260929 10:52:32.437237 1960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.429477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35512 (local address 127.20.80.252:37433)
0929 10:52:32.429513 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:32.429515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:32.429791 (+ 276us) server_negotiation.cc:407] Connection header received
0929 10:52:32.429862 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:32.429863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:32.429874 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:32.429948 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:32.430168 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.430864 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.431788 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.432122 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.432179 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:32.434318 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:32.434319 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:32.434320 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:32.434326 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:32.435771 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:32.436277 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:32.436278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:32.436278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:32.436348 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:32.436553 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:32.436554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:32.436555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:32.436712 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:32.436800 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:32.436970 (+ 170us) server_negotiation.cc:299] Negotiation successful
0929 10:52:32.437126 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Sep 29 10:52:32 dist-test-slave-vqch krb5kdc[1721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.253@KRBTEST.COM
I20260929 10:52:32.438812 1956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0135d6ff59d040f8b68301e68a5faebe: Bootstrap starting.
I20260929 10:52:32.439916 1956 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0135d6ff59d040f8b68301e68a5faebe: Neither blocks nor log segments found. Creating new log.
I20260929 10:52:32.440280 1956 log.cc:824] T 00000000000000000000000000000000 P 0135d6ff59d040f8b68301e68a5faebe: Log is configured to *not* fsync() on all Append() calls
I20260929 10:52:32.441004 1804 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37433 } attempt: 2
I20260929 10:52:32.441152 1956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0135d6ff59d040f8b68301e68a5faebe: No bootstrap required, opened a new log
I20260929 10:52:32.441879 1884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.433046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58082 (local address 127.20.80.253:34087)
0929 10:52:32.433102 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:32.433104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:32.433112 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:52:32.433132 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:32.433132 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:32.433141 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:32.433189 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:32.433784 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.434408 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.435444 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.435624 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.435677 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:32.438535 (+ 2858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:32.438537 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:32.438537 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:32.438544 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:32.440342 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:32.440735 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:32.440736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:32.440737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:32.440792 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:32.441038 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:32.441039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:32.441039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:32.441453 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:32.441654 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:32.441691 (+ 37us) server_negotiation.cc:299] Negotiation successful
0929 10:52:32.441764 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260929 10:52:32.443326 1956 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0135d6ff59d040f8b68301e68a5faebe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8860fcd62a3643409b4308b54568bbb6" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46833 } } peers { permanent_uuid: "0327385e58754995b8e7021dd9821c5e" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 34087 } } peers { permanent_uuid: "0135d6ff59d040f8b68301e68a5faebe" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37433 } }
I20260929 10:52:32.443447 20803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:52:32.443480 1956 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0135d6ff59d040f8b68301e68a5faebe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:52:32.443517 1956 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0135d6ff59d040f8b68301e68a5faebe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0135d6ff59d040f8b68301e68a5faebe, State: Initialized, Role: FOLLOWER
I20260929 10:52:32.443660 1956 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0135d6ff59d040f8b68301e68a5faebe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8860fcd62a3643409b4308b54568bbb6" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46833 } } peers { permanent_uuid: "0327385e58754995b8e7021dd9821c5e" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 34087 } } peers { permanent_uuid: "0135d6ff59d040f8b68301e68a5faebe" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37433 } }
I20260929 10:52:32.443980 1956 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0135d6ff59d040f8b68301e68a5faebe [sys.catalog]: configured and running, proceeding with master startup.
Sep 29 10:52:32 dist-test-slave-vqch krb5kdc[1721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679152, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for kudu/127.20.80.252@KRBTEST.COM
W20260929 10:52:32.445976 1978 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0135d6ff59d040f8b68301e68a5faebe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:52:32.446812 1978 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:52:32.446125 1962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0135d6ff59d040f8b68301e68a5faebe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8860fcd62a3643409b4308b54568bbb6" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46833 } } peers { permanent_uuid: "0327385e58754995b8e7021dd9821c5e" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 34087 } } peers { permanent_uuid: "0135d6ff59d040f8b68301e68a5faebe" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37433 } } }
I20260929 10:52:32.446872 1962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0135d6ff59d040f8b68301e68a5faebe [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:52:32.449004 1806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.441270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:37433 (local address 127.0.0.1:35528)
0929 10:52:32.441321 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:32.441330 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:32.441363 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:32.441531 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:32.441532 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:32.441686 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:52:32.441730 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:32.441732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.442819 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.442821 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:32.443572 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:32.443575 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.444020 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.444021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:32.444087 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:32.444573 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:52:32.444579 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:52:32.446041 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:52:32.447929 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:32.447930 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:32.447930 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:32.448135 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:32.448318 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:32.448319 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:32.448319 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:32.448369 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:32.448664 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:52:32.448665 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:52:32.448720 (+ 55us) client_negotiation.cc:769] Sending connection context
0929 10:52:32.448795 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 10:52:32.448883 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260929 10:52:32.449533 1960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.441303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35528 (local address 127.20.80.252:37433)
0929 10:52:32.441343 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:32.441344 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:32.441355 (+ 11us) server_negotiation.cc:407] Connection header received
0929 10:52:32.441426 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:32.441427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:32.441436 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:32.441798 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:32.441825 (+ 27us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.442715 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.443671 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.443934 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.444006 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:32.446142 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:32.446144 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:32.446144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:32.446152 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:32.447848 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:32.448207 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:32.448208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:32.448208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:32.448258 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:32.448426 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:32.448427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:32.448427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:32.448578 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:32.448637 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:32.449366 (+ 729us) server_negotiation.cc:299] Negotiation successful
0929 10:52:32.449448 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 10:52:32.447674 1887 master_runner.cc:186] Error getting master registration for 127.20.80.254:46833: OK, instance_id { permanent_uuid: "8860fcd62a3643409b4308b54568bbb6" instance_seqno: 1790679152097426 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 10:52:32.451185 1887 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 10:52:32.451197 1804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8860fcd62a3643409b4308b54568bbb6: Bootstrap starting.
I20260929 10:52:32.451915 1804 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8860fcd62a3643409b4308b54568bbb6: Neither blocks nor log segments found. Creating new log.
I20260929 10:52:32.452214 1804 log.cc:824] T 00000000000000000000000000000000 P 8860fcd62a3643409b4308b54568bbb6: Log is configured to *not* fsync() on all Append() calls
I20260929 10:52:32.453083 1804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8860fcd62a3643409b4308b54568bbb6: No bootstrap required, opened a new log
I20260929 10:52:32.454449 1884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.446437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58098 (local address 127.20.80.253:34087)
0929 10:52:32.446480 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:32.446482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:32.446546 (+ 64us) server_negotiation.cc:407] Connection header received
0929 10:52:32.446659 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:32.446660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:32.446671 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:32.446727 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:32.447707 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.448415 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.449438 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.449737 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.449782 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:32.451279 (+ 1497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:32.451281 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:32.451281 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:32.451288 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:32.452971 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:32.453411 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:32.453412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:32.453413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:32.453470 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:32.453672 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:32.453673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:32.453673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:32.453796 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:32.453911 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:32.454280 (+ 369us) server_negotiation.cc:299] Negotiation successful
0929 10:52:32.454361 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 10:52:32.455528 1804 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8860fcd62a3643409b4308b54568bbb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8860fcd62a3643409b4308b54568bbb6" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46833 } } peers { permanent_uuid: "0327385e58754995b8e7021dd9821c5e" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 34087 } } peers { permanent_uuid: "0135d6ff59d040f8b68301e68a5faebe" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37433 } }
I20260929 10:52:32.455672 1804 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8860fcd62a3643409b4308b54568bbb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:52:32.455709 1804 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8860fcd62a3643409b4308b54568bbb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8860fcd62a3643409b4308b54568bbb6, State: Initialized, Role: FOLLOWER
I20260929 10:52:32.455821 1804 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8860fcd62a3643409b4308b54568bbb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8860fcd62a3643409b4308b54568bbb6" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46833 } } peers { permanent_uuid: "0327385e58754995b8e7021dd9821c5e" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 34087 } } peers { permanent_uuid: "0135d6ff59d040f8b68301e68a5faebe" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37433 } }
I20260929 10:52:32.456103 1984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8860fcd62a3643409b4308b54568bbb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8860fcd62a3643409b4308b54568bbb6" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46833 } } peers { permanent_uuid: "0327385e58754995b8e7021dd9821c5e" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 34087 } } peers { permanent_uuid: "0135d6ff59d040f8b68301e68a5faebe" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37433 } } }
I20260929 10:52:32.456140 1804 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8860fcd62a3643409b4308b54568bbb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:52:32.456212 1984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8860fcd62a3643409b4308b54568bbb6 [sys.catalog]: This master's current role is: FOLLOWER
W20260929 10:52:32.459343 1995 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8860fcd62a3643409b4308b54568bbb6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:52:32.459399 1995 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:52:32.460222 1734 master_runner.cc:186] Error getting master registration for 127.20.80.253:34087: OK, instance_id { permanent_uuid: "0327385e58754995b8e7021dd9821c5e" instance_seqno: 1790679152246674 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 10:52:32.463049 1734 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 10:52:32.464284 1807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.445087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44558 (local address 127.20.80.254:46833)
0929 10:52:32.445130 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:32.445132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:32.445234 (+ 102us) server_negotiation.cc:407] Connection header received
0929 10:52:32.445322 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:32.445323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:32.445334 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:32.445417 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:32.446155 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.447160 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.448140 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.448421 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.448463 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:32.457954 (+ 9491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:32.457956 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:32.457957 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:32.457964 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:32.459515 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:32.463162 (+ 3647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:32.463163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:32.463164 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:32.463224 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:32.463453 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:32.463454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:32.463454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:32.463591 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:32.463692 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:32.464122 (+ 430us) server_negotiation.cc:299] Negotiation successful
0929 10:52:32.464196 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 10:52:32.464383 1960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.446571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35536 (local address 127.20.80.252:37433)
0929 10:52:32.449590 (+ 3019us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:32.449591 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:32.452857 (+ 3266us) server_negotiation.cc:407] Connection header received
0929 10:52:32.452952 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:32.452953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:32.452963 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:32.453012 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:32.453716 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.454360 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.455631 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.455914 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.455962 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:32.458273 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:32.458274 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:32.458275 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:32.458281 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:32.459987 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:32.463559 (+ 3572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:32.463560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:32.463561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:32.463618 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:32.463816 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:32.463817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:32.463817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:32.463950 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:32.464025 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:32.464210 (+ 185us) server_negotiation.cc:299] Negotiation successful
0929 10:52:32.464286 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3007,"thread_start_us":68,"threads_started":1}
I20260929 10:52:32.484128 1880 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37433 } attempt: 3
Sep 29 10:52:32 dist-test-slave-vqch krb5kdc[1721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679152, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.253@KRBTEST.COM for kudu/127.20.80.252@KRBTEST.COM
I20260929 10:52:32.492422 1882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.484432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:37433 (local address 127.0.0.1:35538)
0929 10:52:32.484486 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:32.484497 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:32.484534 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:32.484693 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:32.484694 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:32.484917 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:52:32.485000 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:32.485003 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.485960 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.485961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:32.486837 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:32.486840 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.487319 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.487320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:32.487381 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:32.487912 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:52:32.487918 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:52:32.489467 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:52:32.491297 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:32.491299 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:32.491299 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:32.491515 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:32.491742 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:32.491743 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:32.491744 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:32.491811 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:32.492084 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:52:32.492085 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:52:32.492134 (+ 49us) client_negotiation.cc:769] Sending connection context
0929 10:52:32.492209 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 10:52:32.492305 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260929 10:52:32.492426 1960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:32.484512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35538 (local address 127.20.80.252:37433)
0929 10:52:32.484556 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:32.484558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:32.484585 (+ 27us) server_negotiation.cc:407] Connection header received
0929 10:52:32.484608 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:32.484608 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:32.484618 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:32.484680 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:32.485077 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.485858 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.486934 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:32.487240 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:32.487305 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:32.489555 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:32.489556 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:32.489557 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:32.489564 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:32.491215 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:32.491590 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:32.491592 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:32.491592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:32.491652 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:32.491880 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:32.491881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:32.491881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:32.492014 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:32.492083 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:32.492245 (+ 162us) server_negotiation.cc:299] Negotiation successful
0929 10:52:32.492323 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 10:52:32.494462 1880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0327385e58754995b8e7021dd9821c5e: Bootstrap starting.
I20260929 10:52:32.495221 1880 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0327385e58754995b8e7021dd9821c5e: Neither blocks nor log segments found. Creating new log.
I20260929 10:52:32.495527 1880 log.cc:824] T 00000000000000000000000000000000 P 0327385e58754995b8e7021dd9821c5e: Log is configured to *not* fsync() on all Append() calls
I20260929 10:52:32.496467 1880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0327385e58754995b8e7021dd9821c5e: No bootstrap required, opened a new log
I20260929 10:52:32.498600 1880 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0327385e58754995b8e7021dd9821c5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8860fcd62a3643409b4308b54568bbb6" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46833 } } peers { permanent_uuid: "0327385e58754995b8e7021dd9821c5e" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 34087 } } peers { permanent_uuid: "0135d6ff59d040f8b68301e68a5faebe" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37433 } }
I20260929 10:52:32.498755 1880 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0327385e58754995b8e7021dd9821c5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:52:32.498791 1880 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0327385e58754995b8e7021dd9821c5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0327385e58754995b8e7021dd9821c5e, State: Initialized, Role: FOLLOWER
I20260929 10:52:32.498911 1880 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0327385e58754995b8e7021dd9821c5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8860fcd62a3643409b4308b54568bbb6" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46833 } } peers { permanent_uuid: "0327385e58754995b8e7021dd9821c5e" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 34087 } } peers { permanent_uuid: "0135d6ff59d040f8b68301e68a5faebe" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37433 } }
I20260929 10:52:32.499235 1880 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0327385e58754995b8e7021dd9821c5e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:52:32.499185 1998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0327385e58754995b8e7021dd9821c5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8860fcd62a3643409b4308b54568bbb6" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46833 } } peers { permanent_uuid: "0327385e58754995b8e7021dd9821c5e" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 34087 } } peers { permanent_uuid: "0135d6ff59d040f8b68301e68a5faebe" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37433 } } }
I20260929 10:52:32.499392 1998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0327385e58754995b8e7021dd9821c5e [sys.catalog]: This master's current role is: FOLLOWER
W20260929 10:52:32.501046 2009 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0327385e58754995b8e7021dd9821c5e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:52:32.501149 2009 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:52:32.503165 1811 master_runner.cc:263] No leader master found from master 0327385e58754995b8e7021dd9821c5e
I20260929 10:52:32.503216 1811 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 10:52:32.512949 1984 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8860fcd62a3643409b4308b54568bbb6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:52:32.513078 1984 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8860fcd62a3643409b4308b54568bbb6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8860fcd62a3643409b4308b54568bbb6" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46833 } } peers { permanent_uuid: "0327385e58754995b8e7021dd9821c5e" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 34087 } } peers { permanent_uuid: "0135d6ff59d040f8b68301e68a5faebe" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37433 } }
I20260929 10:52:32.513319 1984 leader_election.cc:288] T 00000000000000000000000000000000 P 8860fcd62a3643409b4308b54568bbb6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0327385e58754995b8e7021dd9821c5e (127.20.80.253:34087), 0135d6ff59d040f8b68301e68a5faebe (127.20.80.252:37433)
I20260929 10:52:32.513679 1853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8860fcd62a3643409b4308b54568bbb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0327385e58754995b8e7021dd9821c5e" is_pre_election: true
I20260929 10:52:32.513721 1929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8860fcd62a3643409b4308b54568bbb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0135d6ff59d040f8b68301e68a5faebe" is_pre_election: true
I20260929 10:52:32.513828 1853 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0327385e58754995b8e7021dd9821c5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8860fcd62a3643409b4308b54568bbb6 in term 0.
I20260929 10:52:32.513859 1929 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0135d6ff59d040f8b68301e68a5faebe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8860fcd62a3643409b4308b54568bbb6 in term 0.
I20260929 10:52:32.514465 1751 leader_election.cc:302] T 00000000000000000000000000000000 P 8860fcd62a3643409b4308b54568bbb6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0327385e58754995b8e7021dd9821c5e, 8860fcd62a3643409b4308b54568bbb6; no voters:
I20260929 10:52:32.514981 1984 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8860fcd62a3643409b4308b54568bbb6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:52:32.515051 1984 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8860fcd62a3643409b4308b54568bbb6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:52:32.515081 1984 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8860fcd62a3643409b4308b54568bbb6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:52:32.515991 1984 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8860fcd62a3643409b4308b54568bbb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8860fcd62a3643409b4308b54568bbb6" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46833 } } peers { permanent_uuid: "0327385e58754995b8e7021dd9821c5e" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 34087 } } peers { permanent_uuid: "0135d6ff59d040f8b68301e68a5faebe" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37433 } }
I20260929 10:52:32.516110 1984 leader_election.cc:288] T 00000000000000000000000000000000 P 8860fcd62a3643409b4308b54568bbb6 [CANDIDATE]: Term 1 election: Requested vote from peers 0327385e58754995b8e7021dd9821c5e (127.20.80.253:34087), 0135d6ff59d040f8b68301e68a5faebe (127.20.80.252:37433)
I20260929 10:52:32.516292 1853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8860fcd62a3643409b4308b54568bbb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0327385e58754995b8e7021dd9821c5e"
I20260929 10:52:32.516364 1853 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0327385e58754995b8e7021dd9821c5e [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:52:32.517077 1929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8860fcd62a3643409b4308b54568bbb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0135d6ff59d040f8b68301e68a5faebe"
I20260929 10:52:32.517161 1929 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0135d6ff59d040f8b68301e68a5faebe [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:52:32.517948 1929 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0135d6ff59d040f8b68301e68a5faebe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8860fcd62a3643409b4308b54568bbb6 in term 1.
I20260929 10:52:32.518110 1751 leader_election.cc:302] T 00000000000000000000000000000000 P 8860fcd62a3643409b4308b54568bbb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0135d6ff59d040f8b68301e68a5faebe, 8860fcd62a3643409b4308b54568bbb6; no voters:
I20260929 10:52:32.518491 1853 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0327385e58754995b8e7021dd9821c5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8860fcd62a3643409b4308b54568bbb6 in term 1.
I20260929 10:52:32.518684 1984 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8860fcd62a3643409b4308b54568bbb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:52:32.518853 1984 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8860fcd62a3643409b4308b54568bbb6 [term 1 LEADER]: Becoming Leader. State: Replica: 8860fcd62a3643409b4308b54568bbb6, State: Running, Role: LEADER
I20260929 10:52:32.518957 1984 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8860fcd62a3643409b4308b54568bbb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8860fcd62a3643409b4308b54568bbb6" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46833 } } peers { permanent_uuid: "0327385e58754995b8e7021dd9821c5e" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 34087 } } peers { permanent_uuid: "0135d6ff59d040f8b68301e68a5faebe" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37433 } }
I20260929 10:52:32.519345 1984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8860fcd62a3643409b4308b54568bbb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8860fcd62a3643409b4308b54568bbb6. Latest consensus state: current_term: 1 leader_uuid: "8860fcd62a3643409b4308b54568bbb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8860fcd62a3643409b4308b54568bbb6" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46833 } } peers { permanent_uuid: "0327385e58754995b8e7021dd9821c5e" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 34087 } } peers { permanent_uuid: "0135d6ff59d040f8b68301e68a5faebe" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37433 } } }
I20260929 10:52:32.519428 1984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8860fcd62a3643409b4308b54568bbb6 [sys.catalog]: This master's current role is: LEADER
I20260929 10:52:32.519644 2013 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:52:32.519826 2013 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:52:32.521689 1929 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0135d6ff59d040f8b68301e68a5faebe [term 1 FOLLOWER]: Refusing update from remote peer 8860fcd62a3643409b4308b54568bbb6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:52:32.521983 1984 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8860fcd62a3643409b4308b54568bbb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0135d6ff59d040f8b68301e68a5faebe" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:52:32.523186 1962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0135d6ff59d040f8b68301e68a5faebe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8860fcd62a3643409b4308b54568bbb6. Latest consensus state: current_term: 1 leader_uuid: "8860fcd62a3643409b4308b54568bbb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8860fcd62a3643409b4308b54568bbb6" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46833 } } peers { permanent_uuid: "0327385e58754995b8e7021dd9821c5e" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 34087 } } peers { permanent_uuid: "0135d6ff59d040f8b68301e68a5faebe" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37433 } } }
I20260929 10:52:32.523290 1962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0135d6ff59d040f8b68301e68a5faebe [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:52:32.524349 1853 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0327385e58754995b8e7021dd9821c5e [term 1 FOLLOWER]: Refusing update from remote peer 8860fcd62a3643409b4308b54568bbb6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:52:32.524591 1984 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8860fcd62a3643409b4308b54568bbb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0327385e58754995b8e7021dd9821c5e" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 34087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:52:32.525832 1998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0327385e58754995b8e7021dd9821c5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8860fcd62a3643409b4308b54568bbb6. Latest consensus state: current_term: 1 leader_uuid: "8860fcd62a3643409b4308b54568bbb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8860fcd62a3643409b4308b54568bbb6" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46833 } } peers { permanent_uuid: "0327385e58754995b8e7021dd9821c5e" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 34087 } } peers { permanent_uuid: "0135d6ff59d040f8b68301e68a5faebe" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37433 } } }
I20260929 10:52:32.525942 1998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0327385e58754995b8e7021dd9821c5e [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:52:32.527020 2014 mvcc.cc:205] Tried to move back new op lower bound from 7334621808726634496 to 7334621808718184448. Current Snapshot: MvccSnapshot[applied={T|T < 7334621808726634496}]
I20260929 10:52:32.527309 1984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8860fcd62a3643409b4308b54568bbb6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8860fcd62a3643409b4308b54568bbb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8860fcd62a3643409b4308b54568bbb6" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46833 } } peers { permanent_uuid: "0327385e58754995b8e7021dd9821c5e" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 34087 } } peers { permanent_uuid: "0135d6ff59d040f8b68301e68a5faebe" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37433 } } }
I20260929 10:52:32.527391 1984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8860fcd62a3643409b4308b54568bbb6 [sys.catalog]: This master's current role is: LEADER
I20260929 10:52:32.527777 1962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0135d6ff59d040f8b68301e68a5faebe [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8860fcd62a3643409b4308b54568bbb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8860fcd62a3643409b4308b54568bbb6" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46833 } } peers { permanent_uuid: "0327385e58754995b8e7021dd9821c5e" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 34087 } } peers { permanent_uuid: "0135d6ff59d040f8b68301e68a5faebe" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37433 } } }
I20260929 10:52:32.527864 1962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0135d6ff59d040f8b68301e68a5faebe [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:52:32.528502 2015 mvcc.cc:205] Tried to move back new op lower bound from 7334621808726634496 to 7334621808718184448. Current Snapshot: MvccSnapshot[applied={T|T < 7334621808726634496}]
I20260929 10:52:32.529712 1984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8860fcd62a3643409b4308b54568bbb6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8860fcd62a3643409b4308b54568bbb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8860fcd62a3643409b4308b54568bbb6" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46833 } } peers { permanent_uuid: "0327385e58754995b8e7021dd9821c5e" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 34087 } } peers { permanent_uuid: "0135d6ff59d040f8b68301e68a5faebe" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37433 } } }
I20260929 10:52:32.529801 1984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8860fcd62a3643409b4308b54568bbb6 [sys.catalog]: This master's current role is: LEADER
I20260929 10:52:32.530234 1998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0327385e58754995b8e7021dd9821c5e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8860fcd62a3643409b4308b54568bbb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8860fcd62a3643409b4308b54568bbb6" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 46833 } } peers { permanent_uuid: "0327385e58754995b8e7021dd9821c5e" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 34087 } } peers { permanent_uuid: "0135d6ff59d040f8b68301e68a5faebe" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37433 } } }
I20260929 10:52:32.530325 1998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0327385e58754995b8e7021dd9821c5e [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:52:32.530578 2017 mvcc.cc:205] Tried to move back new op lower bound from 7334621808726634496 to 7334621808718184448. Current Snapshot: MvccSnapshot[applied={T|T < 7334621808726634496}]
I20260929 10:52:32.531150 2013 catalog_manager.cc:1347] Generated new cluster ID: 7efd55dcbbc841ba8ec85098636ccb39
I20260929 10:52:32.531208 2013 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:52:32.548772 2013 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:52:32.549510 2013 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:52:32.566367 2013 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8860fcd62a3643409b4308b54568bbb6: Generated new TSK 0
I20260929 10:52:32.566591 2013 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:52:33.447295 1978 catalog_manager.cc:1259] Loaded cluster ID: 7efd55dcbbc841ba8ec85098636ccb39
I20260929 10:52:33.447376 1978 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0135d6ff59d040f8b68301e68a5faebe: loading cluster ID for follower catalog manager: success
I20260929 10:52:33.448243 1978 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0135d6ff59d040f8b68301e68a5faebe: acquiring CA information for follower catalog manager: success
I20260929 10:52:33.448598 1978 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0135d6ff59d040f8b68301e68a5faebe: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 10:52:33.501637 2009 catalog_manager.cc:1259] Loaded cluster ID: 7efd55dcbbc841ba8ec85098636ccb39
I20260929 10:52:33.501717 2009 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0327385e58754995b8e7021dd9821c5e: loading cluster ID for follower catalog manager: success
I20260929 10:52:33.502558 2009 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0327385e58754995b8e7021dd9821c5e: acquiring CA information for follower catalog manager: success
I20260929 10:52:33.502892 2009 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0327385e58754995b8e7021dd9821c5e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 10:52:33.634363 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 1734
I20260929 10:52:33.640079 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 1811
I20260929 10:52:33.645292 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 1887
2026-09-29T10:52:33Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2843 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:52:33 dist-test-slave-vqch krb5kdc[2031](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:52:33 dist-test-slave-vqch krb5kdc[2031](info): set up 2 sockets
Sep 29 10:52:33 dist-test-slave-vqch krb5kdc[2031](info): commencing operation
krb5kdc: starting...
W20260929 10:52:34.684816 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:52:34 dist-test-slave-vqch krb5kdc[2031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:52:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:52:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.80.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
I20260929 10:52:34.808825 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:33121
--webserver_interface=127.20.80.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:44523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:33121
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:52:34.891863 2044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:52:34.892074 2044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:52:34.892098 2044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:52:34.893266 2044 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:52:34.893303 2044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:52:34.893317 2044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:52:34.893330 2044 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:52:34.893342 2044 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:52:34.895143 2044 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:44523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:33121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:33121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:52:34.895370 2044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:52:34.895550 2044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:52:34.900723 2052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:34.900758 2049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:34.900791 2050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:34.900931 2044 server_base.cc:1034] running on GCE node
I20260929 10:52:34.901263 2044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:52:34.901568 2044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:52:34.902746 2044 hybrid_clock.cc:648] HybridClock initialized: now 1790679154902723 us; error 43 us; skew 500 ppm
Sep 29 10:52:34 dist-test-slave-vqch krb5kdc[2031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679154, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:52:34.905409 2044 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:52:34.905906 2044 webserver.cc:460] Webserver started at http://127.20.80.254:43167/ using document root <none> and password file <none>
I20260929 10:52:34.906157 2044 fs_manager.cc:359] Metadata directory not provided
I20260929 10:52:34.906203 2044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:52:34.906359 2044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:52:34.907440 2044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "cd99a97a085e4690af7f83d1fc454ece"
format_stamp: "Formatted at 2026-09-29 10:52:34 on dist-test-slave-vqch"
server_key: "2133a4c0f244ab4d9cf215a6ab38f5b7"
server_key_iv: "33bea1d5e3eaef68f46f76d24e7367c5"
server_key_version: "encryptionkey@0"
I20260929 10:52:34.907809 2044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "cd99a97a085e4690af7f83d1fc454ece"
format_stamp: "Formatted at 2026-09-29 10:52:34 on dist-test-slave-vqch"
server_key: "2133a4c0f244ab4d9cf215a6ab38f5b7"
server_key_iv: "33bea1d5e3eaef68f46f76d24e7367c5"
server_key_version: "encryptionkey@0"
I20260929 10:52:34.909245 2044 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 10:52:34.910082 2059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:34.910404 2044 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:52:34.910501 2044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "cd99a97a085e4690af7f83d1fc454ece"
format_stamp: "Formatted at 2026-09-29 10:52:34 on dist-test-slave-vqch"
server_key: "2133a4c0f244ab4d9cf215a6ab38f5b7"
server_key_iv: "33bea1d5e3eaef68f46f76d24e7367c5"
server_key_version: "encryptionkey@0"
I20260929 10:52:34.910576 2044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:52:34.926890 2044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:52:34.928929 2044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:52:34.929174 2044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:52:34.933703 2044 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:33121
I20260929 10:52:34.933931 2112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:33121 every 8 connection(s)
I20260929 10:52:34.934167 2044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:52:34.935057 2113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:52:34.937435 2113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd99a97a085e4690af7f83d1fc454ece: Bootstrap starting.
I20260929 10:52:34.938033 2113 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd99a97a085e4690af7f83d1fc454ece: Neither blocks nor log segments found. Creating new log.
I20260929 10:52:34.938325 2113 log.cc:824] T 00000000000000000000000000000000 P cd99a97a085e4690af7f83d1fc454ece: Log is configured to *not* fsync() on all Append() calls
I20260929 10:52:34.939033 2113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd99a97a085e4690af7f83d1fc454ece: No bootstrap required, opened a new log
I20260929 10:52:34.940613 2113 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd99a97a085e4690af7f83d1fc454ece [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd99a97a085e4690af7f83d1fc454ece" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 33121 } }
I20260929 10:52:34.940754 2113 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd99a97a085e4690af7f83d1fc454ece [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:52:34.940778 2113 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd99a97a085e4690af7f83d1fc454ece [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd99a97a085e4690af7f83d1fc454ece, State: Initialized, Role: FOLLOWER
I20260929 10:52:34.940955 2113 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd99a97a085e4690af7f83d1fc454ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd99a97a085e4690af7f83d1fc454ece" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 33121 } }
I20260929 10:52:34.941078 2113 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd99a97a085e4690af7f83d1fc454ece [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:52:34.941121 2113 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd99a97a085e4690af7f83d1fc454ece [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:52:34.941172 2113 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd99a97a085e4690af7f83d1fc454ece [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:52:34.941702 2113 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd99a97a085e4690af7f83d1fc454ece [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd99a97a085e4690af7f83d1fc454ece" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 33121 } }
I20260929 10:52:34.941817 2113 leader_election.cc:302] T 00000000000000000000000000000000 P cd99a97a085e4690af7f83d1fc454ece [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd99a97a085e4690af7f83d1fc454ece; no voters:
I20260929 10:52:34.942011 2113 leader_election.cc:288] T 00000000000000000000000000000000 P cd99a97a085e4690af7f83d1fc454ece [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:52:34.942050 2116 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd99a97a085e4690af7f83d1fc454ece [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:52:34.942234 2116 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd99a97a085e4690af7f83d1fc454ece [term 1 LEADER]: Becoming Leader. State: Replica: cd99a97a085e4690af7f83d1fc454ece, State: Running, Role: LEADER
I20260929 10:52:34.942265 2113 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd99a97a085e4690af7f83d1fc454ece [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:52:34.942327 2116 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd99a97a085e4690af7f83d1fc454ece [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd99a97a085e4690af7f83d1fc454ece" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 33121 } }
I20260929 10:52:34.942734 2117 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd99a97a085e4690af7f83d1fc454ece [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd99a97a085e4690af7f83d1fc454ece" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd99a97a085e4690af7f83d1fc454ece" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 33121 } } }
I20260929 10:52:34.942749 2118 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd99a97a085e4690af7f83d1fc454ece [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd99a97a085e4690af7f83d1fc454ece. Latest consensus state: current_term: 1 leader_uuid: "cd99a97a085e4690af7f83d1fc454ece" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd99a97a085e4690af7f83d1fc454ece" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 33121 } } }
I20260929 10:52:34.942847 2117 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd99a97a085e4690af7f83d1fc454ece [sys.catalog]: This master's current role is: LEADER
I20260929 10:52:34.942852 2118 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd99a97a085e4690af7f83d1fc454ece [sys.catalog]: This master's current role is: LEADER
I20260929 10:52:34.943040 2120 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:52:34.943588 2120 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:52:34.944242 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 2044
I20260929 10:52:34.944367 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:52:34.944584 20803 external_mini_cluster.cc:1428] Setting key 0b198eead86e8167b6d83f8c8112df9d
I20260929 10:52:34.945243 2120 catalog_manager.cc:1347] Generated new cluster ID: 1e7dd0aac9b146d687346ab1c4a0c957
I20260929 10:52:34.945304 2120 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 10:52:34 dist-test-slave-vqch krb5kdc[2031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:52:34.960696 2136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:34.945021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47168 (local address 127.20.80.254:33121)
0929 10:52:34.949305 (+ 4284us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:34.949314 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:34.949352 (+ 38us) server_negotiation.cc:407] Connection header received
0929 10:52:34.949463 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:34.949470 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:34.949521 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:34.949640 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:34.950468 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:34.952133 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:34.953271 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:34.953849 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:34.953955 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:34.956842 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:34.956847 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:34.956847 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:34.956862 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:34.958970 (+ 2108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:34.959378 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:34.959381 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:34.959381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:34.959503 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:34.959708 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:34.959709 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:34.959709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:34.960045 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:34.960202 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:34.960279 (+ 77us) server_negotiation.cc:299] Negotiation successful
0929 10:52:34.960426 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4238,"thread_start_us":66,"threads_started":1}
I20260929 10:52:34.961755 20803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:52:34.963867 2120 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:52:34.964385 2120 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:52:34.971122 2120 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd99a97a085e4690af7f83d1fc454ece: Generated new TSK 0
I20260929 10:52:34.971313 2120 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:52:35.016093 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 2044
2026-09-29T10:52:35Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1369 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:52:35 dist-test-slave-vqch krb5kdc[2140](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:52:35 dist-test-slave-vqch krb5kdc[2140](info): set up 2 sockets
Sep 29 10:52:35 dist-test-slave-vqch krb5kdc[2140](info): commencing operation
krb5kdc: starting...
W20260929 10:52:36.072821 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:52:36 dist-test-slave-vqch krb5kdc[2140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:52:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:52:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.80.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
I20260929 10:52:36.195739 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:37853
--webserver_interface=127.20.80.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:33539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:37853
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:52:36.276672 2153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:52:36.276927 2153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:52:36.276969 2153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:52:36.278151 2153 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:52:36.278223 2153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:52:36.278247 2153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:52:36.278275 2153 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:52:36.278303 2153 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:52:36.279799 2153 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:33539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:37853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:37853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:52:36.280088 2153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:52:36.280333 2153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:52:36.285327 2161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:36.285279 2159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:36.285296 2158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:36.285583 2153 server_base.cc:1034] running on GCE node
I20260929 10:52:36.285863 2153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:52:36.286149 2153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:52:36.287325 2153 hybrid_clock.cc:648] HybridClock initialized: now 1790679156287305 us; error 51 us; skew 500 ppm
Sep 29 10:52:36 dist-test-slave-vqch krb5kdc[2140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679156, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:52:36.290000 2153 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:52:36.290433 2153 webserver.cc:460] Webserver started at http://127.20.80.254:33955/ using document root <none> and password file <none>
I20260929 10:52:36.290673 2153 fs_manager.cc:359] Metadata directory not provided
I20260929 10:52:36.290748 2153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:52:36.290898 2153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:52:36.291952 2153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "1ab3cc1989514145aad5dfa61e847b96"
format_stamp: "Formatted at 2026-09-29 10:52:36 on dist-test-slave-vqch"
server_key: "61726378975ad41bd47ed6428652b572"
server_key_iv: "a271787484b42854562d28a652b506e3"
server_key_version: "encryptionkey@0"
I20260929 10:52:36.292313 2153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "1ab3cc1989514145aad5dfa61e847b96"
format_stamp: "Formatted at 2026-09-29 10:52:36 on dist-test-slave-vqch"
server_key: "61726378975ad41bd47ed6428652b572"
server_key_iv: "a271787484b42854562d28a652b506e3"
server_key_version: "encryptionkey@0"
I20260929 10:52:36.293700 2153 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 10:52:36.294464 2168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:36.294629 2153 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:52:36.294726 2153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "1ab3cc1989514145aad5dfa61e847b96"
format_stamp: "Formatted at 2026-09-29 10:52:36 on dist-test-slave-vqch"
server_key: "61726378975ad41bd47ed6428652b572"
server_key_iv: "a271787484b42854562d28a652b506e3"
server_key_version: "encryptionkey@0"
I20260929 10:52:36.294842 2153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:52:36.329118 2153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:52:36.331555 2153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:52:36.331723 2153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:52:36.335856 2153 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:37853
I20260929 10:52:36.335930 2220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:37853 every 8 connection(s)
I20260929 10:52:36.336282 2153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:52:36.337337 2221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:52:36.339641 2221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96: Bootstrap starting.
I20260929 10:52:36.340245 2221 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96: Neither blocks nor log segments found. Creating new log.
I20260929 10:52:36.340545 2221 log.cc:824] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96: Log is configured to *not* fsync() on all Append() calls
I20260929 10:52:36.341142 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 2153
I20260929 10:52:36.341253 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:52:36.341378 2221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96: No bootstrap required, opened a new log
I20260929 10:52:36.341487 20803 external_mini_cluster.cc:1428] Setting key 4b584952bd70fe31fe54fc68ac789f58
I20260929 10:52:36.343379 2221 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab3cc1989514145aad5dfa61e847b96" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37853 } }
I20260929 10:52:36.343518 2221 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:52:36.343567 2221 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ab3cc1989514145aad5dfa61e847b96, State: Initialized, Role: FOLLOWER
I20260929 10:52:36.343721 2221 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab3cc1989514145aad5dfa61e847b96" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37853 } }
I20260929 10:52:36.343791 2221 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:52:36.343813 2221 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:52:36.343880 2221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:52:36.344444 2221 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab3cc1989514145aad5dfa61e847b96" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37853 } }
I20260929 10:52:36.344532 2221 leader_election.cc:302] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ab3cc1989514145aad5dfa61e847b96; no voters:
I20260929 10:52:36.344724 2221 leader_election.cc:288] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:52:36.344745 2226 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:52:36.344853 2226 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [term 1 LEADER]: Becoming Leader. State: Replica: 1ab3cc1989514145aad5dfa61e847b96, State: Running, Role: LEADER
I20260929 10:52:36.344939 2226 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab3cc1989514145aad5dfa61e847b96" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37853 } }
I20260929 10:52:36.345027 2221 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:52:36.345315 2227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ab3cc1989514145aad5dfa61e847b96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab3cc1989514145aad5dfa61e847b96" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37853 } } }
I20260929 10:52:36.345408 2227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [sys.catalog]: This master's current role is: LEADER
I20260929 10:52:36.345574 2228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ab3cc1989514145aad5dfa61e847b96. Latest consensus state: current_term: 1 leader_uuid: "1ab3cc1989514145aad5dfa61e847b96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab3cc1989514145aad5dfa61e847b96" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37853 } } }
I20260929 10:52:36.345631 2228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [sys.catalog]: This master's current role is: LEADER
I20260929 10:52:36.345693 2231 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:52:36.346256 2231 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:52:36.348337 2231 catalog_manager.cc:1347] Generated new cluster ID: 52e1580f6b814e2d8b2a82cec0f7b7e5
I20260929 10:52:36.348404 2231 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 10:52:36 dist-test-slave-vqch krb5kdc[2140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:52:36.352692 2225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:36.342022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35082 (local address 127.20.80.254:37853)
0929 10:52:36.342222 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:36.342232 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:36.342272 (+ 40us) server_negotiation.cc:407] Connection header received
0929 10:52:36.342696 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:36.342703 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:36.342748 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:36.342896 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:36.343507 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.344929 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.346033 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.346725 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.346845 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:36.349417 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:36.349421 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:36.349421 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:36.349433 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:36.351121 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:36.351494 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:36.351496 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:36.351497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:36.351584 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:36.351798 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:36.351799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:36.351800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:36.352030 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:36.352200 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:36.352304 (+ 104us) server_negotiation.cc:299] Negotiation successful
0929 10:52:36.352546 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":99,"threads_started":1}
I20260929 10:52:36.384433 2231 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:52:36.385070 2231 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:52:36.388121 2231 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96: Generated new TSK 0
I20260929 10:52:36.388316 2231 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.80.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
I20260929 10:52:36.424491 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.193:0
--local_ip_for_outbound_sockets=127.20.80.193
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37853
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:33539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:52:36.521200 2248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:52:36.521395 2248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:52:36.521418 2248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:52:36.522476 2248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:52:36.522509 2248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:52:36.522534 2248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.193
I20260929 10:52:36.523875 2248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:33539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.193
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:52:36.524060 2248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:52:36.524217 2248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:52:36.529383 2253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:36.529403 2254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:36.529403 2256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:36.529433 2248 server_base.cc:1034] running on GCE node
I20260929 10:52:36.529750 2248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:52:36.529946 2248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:52:36.531077 2248 hybrid_clock.cc:648] HybridClock initialized: now 1790679156531062 us; error 26 us; skew 500 ppm
Sep 29 10:52:36 dist-test-slave-vqch krb5kdc[2140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679156, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:52:36.533607 2248 init.cc:377] Logged in from keytab as kudu/127.20.80.193@KRBTEST.COM (short username kudu)
I20260929 10:52:36.534075 2248 webserver.cc:460] Webserver started at http://127.20.80.193:46475/ using document root <none> and password file <none>
I20260929 10:52:36.534315 2248 fs_manager.cc:359] Metadata directory not provided
I20260929 10:52:36.534361 2248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:52:36.534511 2248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:52:36.535547 2248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/data/instance:
uuid: "6b59f952e57f4451904f3534100a4435"
format_stamp: "Formatted at 2026-09-29 10:52:36 on dist-test-slave-vqch"
server_key: "6bcb6adfd5e3334122d7e4e26f49e70d"
server_key_iv: "ca704924dffe88c3936aaf7d35064945"
server_key_version: "encryptionkey@0"
I20260929 10:52:36.535912 2248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance:
uuid: "6b59f952e57f4451904f3534100a4435"
format_stamp: "Formatted at 2026-09-29 10:52:36 on dist-test-slave-vqch"
server_key: "6bcb6adfd5e3334122d7e4e26f49e70d"
server_key_iv: "ca704924dffe88c3936aaf7d35064945"
server_key_version: "encryptionkey@0"
I20260929 10:52:36.537273 2248 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 10:52:36.538012 2263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:36.538183 2248 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:52:36.538281 2248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/wal
uuid: "6b59f952e57f4451904f3534100a4435"
format_stamp: "Formatted at 2026-09-29 10:52:36 on dist-test-slave-vqch"
server_key: "6bcb6adfd5e3334122d7e4e26f49e70d"
server_key_iv: "ca704924dffe88c3936aaf7d35064945"
server_key_version: "encryptionkey@0"
I20260929 10:52:36.538360 2248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:52:36.547149 2248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:52:36.548846 2248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:52:36.549060 2248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:52:36.549278 2248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:52:36.549572 2248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:52:36.549625 2248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:36.549665 2248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:52:36.549691 2248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:36.555478 2248 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.193:43659
I20260929 10:52:36.555528 2376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.193:43659 every 8 connection(s)
I20260929 10:52:36.555822 2248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
I20260929 10:52:36.560170 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 2248
I20260929 10:52:36.560276 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance
I20260929 10:52:36.560458 20803 external_mini_cluster.cc:1428] Setting key 41e140f5ffc9196b08fdcec84563cd27
Sep 29 10:52:36 dist-test-slave-vqch krb5kdc[2140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679156, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:52:36.566433 2225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:36.556545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:50545 (local address 127.20.80.254:37853)
0929 10:52:36.556617 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:36.556619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:36.556869 (+ 250us) server_negotiation.cc:407] Connection header received
0929 10:52:36.557356 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:36.557357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:36.557367 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:36.557441 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:36.558366 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.558884 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.559891 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.560199 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.560280 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:36.562637 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:36.562639 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:36.562640 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:36.562648 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:36.564551 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:36.565082 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:36.565083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:36.565084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:36.565177 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:36.565546 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:36.565547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:36.565548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:36.565705 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:36.565781 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:36.566192 (+ 411us) server_negotiation.cc:299] Negotiation successful
0929 10:52:36.566292 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 10:52:36.567011 2380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:36.556536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.193:50545)
0929 10:52:36.556736 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:36.556778 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:36.557273 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:36.557516 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:36.557519 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:36.558104 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:52:36.558287 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:36.558292 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.558988 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.558989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:36.559813 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:36.559816 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.560311 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.560312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:36.560461 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:36.561107 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:52:36.561118 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:52:36.562537 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:52:36.564643 (+ 2106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:36.564652 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:36.564654 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:36.564958 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:36.565289 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:36.565290 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:36.565290 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:36.565463 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:36.565814 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:52:36.565822 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:52:36.565996 (+ 174us) client_negotiation.cc:769] Sending connection context
0929 10:52:36.566539 (+ 543us) client_negotiation.cc:240] Negotiation successful
0929 10:52:36.566728 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":102,"threads_started":1}
I20260929 10:52:36.567615 2377 heartbeater.cc:347] Connected to a master server at 127.20.80.254:37853
I20260929 10:52:36.567726 2377 heartbeater.cc:464] Registering TS with master...
I20260929 10:52:36.567934 2377 heartbeater.cc:511] Master 127.20.80.254:37853 requested a full tablet report, sending...
I20260929 10:52:36.568483 2185 ts_manager.cc:194] Registered new tserver with Master: 6b59f952e57f4451904f3534100a4435 (127.20.80.193:43659)
I20260929 10:52:36.569186 2185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.193@KRBTEST.COM'} at 127.20.80.193:50545
WARNING: no policy specified for kudu/127.20.80.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
I20260929 10:52:36.588495 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.194:0
--local_ip_for_outbound_sockets=127.20.80.194
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37853
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:33539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:52:36.668479 2383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:52:36.668681 2383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:52:36.668705 2383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:52:36.669925 2383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:52:36.669970 2383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:52:36.669996 2383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.194
I20260929 10:52:36.671450 2383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:33539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.194
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:52:36.671698 2383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:52:36.671926 2383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:52:36.677192 2391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:36.677170 2388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:36.677172 2389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:36.677203 2383 server_base.cc:1034] running on GCE node
I20260929 10:52:36.677620 2383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:52:36.677861 2383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:52:36.679037 2383 hybrid_clock.cc:648] HybridClock initialized: now 1790679156679023 us; error 27 us; skew 500 ppm
Sep 29 10:52:36 dist-test-slave-vqch krb5kdc[2140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679156, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:52:36.681536 2383 init.cc:377] Logged in from keytab as kudu/127.20.80.194@KRBTEST.COM (short username kudu)
I20260929 10:52:36.681989 2383 webserver.cc:460] Webserver started at http://127.20.80.194:35347/ using document root <none> and password file <none>
I20260929 10:52:36.682224 2383 fs_manager.cc:359] Metadata directory not provided
I20260929 10:52:36.682293 2383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:52:36.682449 2383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:52:36.683565 2383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/data/instance:
uuid: "b03c7542574e48749b01c123a1a93661"
format_stamp: "Formatted at 2026-09-29 10:52:36 on dist-test-slave-vqch"
server_key: "65b0caa4c3eea46eeb1479b88b780f00"
server_key_iv: "2683f4a25f1de177be466753e13f1ce5"
server_key_version: "encryptionkey@0"
I20260929 10:52:36.683933 2383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance:
uuid: "b03c7542574e48749b01c123a1a93661"
format_stamp: "Formatted at 2026-09-29 10:52:36 on dist-test-slave-vqch"
server_key: "65b0caa4c3eea46eeb1479b88b780f00"
server_key_iv: "2683f4a25f1de177be466753e13f1ce5"
server_key_version: "encryptionkey@0"
I20260929 10:52:36.685293 2383 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 10:52:36.686017 2398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:36.686180 2383 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:52:36.686277 2383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/wal
uuid: "b03c7542574e48749b01c123a1a93661"
format_stamp: "Formatted at 2026-09-29 10:52:36 on dist-test-slave-vqch"
server_key: "65b0caa4c3eea46eeb1479b88b780f00"
server_key_iv: "2683f4a25f1de177be466753e13f1ce5"
server_key_version: "encryptionkey@0"
I20260929 10:52:36.686353 2383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:52:36.699421 2383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:52:36.701470 2383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:52:36.701642 2383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:52:36.701881 2383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:52:36.702190 2383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:52:36.702245 2383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:36.702287 2383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:52:36.702310 2383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:36.708171 2383 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.194:34205
I20260929 10:52:36.708222 2511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.194:34205 every 8 connection(s)
I20260929 10:52:36.708530 2383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
I20260929 10:52:36.713428 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 2383
I20260929 10:52:36.713527 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance
I20260929 10:52:36.713729 20803 external_mini_cluster.cc:1428] Setting key 4f9ae08ee9c48e44c13e5392a152252a
Sep 29 10:52:36 dist-test-slave-vqch krb5kdc[2140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679156, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:52:36.717957 2225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:36.709268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:53521 (local address 127.20.80.254:37853)
0929 10:52:36.709345 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:36.709348 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:36.709511 (+ 163us) server_negotiation.cc:407] Connection header received
0929 10:52:36.709975 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:36.709976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:36.709986 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:36.710038 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:36.710919 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.711426 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.712365 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.712564 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.712619 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:36.715051 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:36.715053 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:36.715053 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:36.715062 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:36.716521 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:36.716946 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:36.716948 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:36.716949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:36.717057 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:36.717315 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:36.717315 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:36.717316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:36.717436 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:36.717514 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:36.717783 (+ 269us) server_negotiation.cc:299] Negotiation successful
0929 10:52:36.717865 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 10:52:36.718145 2514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:36.709221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.194:53521)
0929 10:52:36.709381 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:36.709423 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:36.709888 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:36.710091 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:36.710094 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:36.710645 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:52:36.710837 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:36.710841 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.711508 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.711508 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:36.712283 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:36.712285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.712640 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.712641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:36.712741 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:36.713180 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:52:36.713187 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:52:36.714882 (+ 1695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:52:36.716608 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:36.716616 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:36.716619 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:36.716849 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:36.717123 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:36.717123 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:36.717124 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:36.717244 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:36.717504 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:52:36.717511 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:52:36.717650 (+ 139us) client_negotiation.cc:769] Sending connection context
0929 10:52:36.717761 (+ 111us) client_negotiation.cc:240] Negotiation successful
0929 10:52:36.717920 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":98,"threads_started":1}
I20260929 10:52:36.718600 2512 heartbeater.cc:347] Connected to a master server at 127.20.80.254:37853
I20260929 10:52:36.718709 2512 heartbeater.cc:464] Registering TS with master...
I20260929 10:52:36.718900 2512 heartbeater.cc:511] Master 127.20.80.254:37853 requested a full tablet report, sending...
I20260929 10:52:36.719341 2185 ts_manager.cc:194] Registered new tserver with Master: b03c7542574e48749b01c123a1a93661 (127.20.80.194:34205)
I20260929 10:52:36.719726 2185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.194@KRBTEST.COM'} at 127.20.80.194:53521
WARNING: no policy specified for kudu/127.20.80.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
I20260929 10:52:36.740216 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.195:0
--local_ip_for_outbound_sockets=127.20.80.195
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37853
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:33539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:52:36.822165 2517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:52:36.822363 2517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:52:36.822386 2517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:52:36.823475 2517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:52:36.823513 2517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:52:36.823539 2517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.195
I20260929 10:52:36.824990 2517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:33539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.195
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:52:36.825165 2517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:52:36.825344 2517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:52:36.830135 2525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:36.830175 2522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:36.830144 2523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:36.830816 2517 server_base.cc:1034] running on GCE node
I20260929 10:52:36.831056 2517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:52:36.831305 2517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:52:36.832476 2517 hybrid_clock.cc:648] HybridClock initialized: now 1790679156832463 us; error 43 us; skew 500 ppm
Sep 29 10:52:36 dist-test-slave-vqch krb5kdc[2140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679156, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:52:36.834968 2517 init.cc:377] Logged in from keytab as kudu/127.20.80.195@KRBTEST.COM (short username kudu)
I20260929 10:52:36.835351 2517 webserver.cc:460] Webserver started at http://127.20.80.195:34129/ using document root <none> and password file <none>
I20260929 10:52:36.835526 2517 fs_manager.cc:359] Metadata directory not provided
I20260929 10:52:36.835595 2517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:52:36.835736 2517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:52:36.836707 2517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/data/instance:
uuid: "640879e300bf411c96666407655e9985"
format_stamp: "Formatted at 2026-09-29 10:52:36 on dist-test-slave-vqch"
server_key: "cc356bc2c12fcda8cb3e86049eba111e"
server_key_iv: "313662a96148ee583226440d8020aacd"
server_key_version: "encryptionkey@0"
I20260929 10:52:36.837057 2517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance:
uuid: "640879e300bf411c96666407655e9985"
format_stamp: "Formatted at 2026-09-29 10:52:36 on dist-test-slave-vqch"
server_key: "cc356bc2c12fcda8cb3e86049eba111e"
server_key_iv: "313662a96148ee583226440d8020aacd"
server_key_version: "encryptionkey@0"
I20260929 10:52:36.838325 2517 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:52:36.839049 2532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:36.839284 2517 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:52:36.839357 2517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/wal
uuid: "640879e300bf411c96666407655e9985"
format_stamp: "Formatted at 2026-09-29 10:52:36 on dist-test-slave-vqch"
server_key: "cc356bc2c12fcda8cb3e86049eba111e"
server_key_iv: "313662a96148ee583226440d8020aacd"
server_key_version: "encryptionkey@0"
I20260929 10:52:36.839449 2517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:52:36.861547 2517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:52:36.863379 2517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:52:36.863536 2517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:52:36.863770 2517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:52:36.864084 2517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:52:36.864137 2517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:36.864179 2517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:52:36.864205 2517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:36.869902 2517 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.195:39199
I20260929 10:52:36.869994 2647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.195:39199 every 8 connection(s)
I20260929 10:52:36.870293 2517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
I20260929 10:52:36.875367 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 2517
I20260929 10:52:36.875475 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance
I20260929 10:52:36.875653 20803 external_mini_cluster.cc:1428] Setting key e61f41e8eb05e782e114ac2eb4903b34
Sep 29 10:52:36 dist-test-slave-vqch krb5kdc[2140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679156, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:52:36.879937 2225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:36.871106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:52685 (local address 127.20.80.254:37853)
0929 10:52:36.871194 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:36.871196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:36.871360 (+ 164us) server_negotiation.cc:407] Connection header received
0929 10:52:36.871845 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:36.871846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:36.871857 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:36.871918 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:36.872837 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.873407 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.874418 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.874610 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.874663 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:36.877331 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:36.877332 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:36.877333 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:36.877340 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:36.878550 (+ 1210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:36.878924 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:36.878925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:36.878925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:36.878994 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:36.879235 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:36.879236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:36.879236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:36.879355 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:36.879420 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:36.879773 (+ 353us) server_negotiation.cc:299] Negotiation successful
0929 10:52:36.879874 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 10:52:36.880092 2650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:36.871070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.195:52685)
0929 10:52:36.871234 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:36.871276 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:36.871753 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:36.871967 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:36.871970 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:36.872548 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:52:36.872738 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:36.872743 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.873498 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.873499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:36.874303 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:36.874306 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.874684 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.874685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:36.874786 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:36.875335 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:52:36.875343 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:52:36.877241 (+ 1898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:52:36.878624 (+ 1383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:36.878631 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:36.878632 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:36.878838 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:36.879061 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:36.879062 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:36.879062 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:36.879154 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:36.879423 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:52:36.879447 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:52:36.879639 (+ 192us) client_negotiation.cc:769] Sending connection context
0929 10:52:36.879752 (+ 113us) client_negotiation.cc:240] Negotiation successful
0929 10:52:36.879878 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":96,"threads_started":1}
I20260929 10:52:36.880511 2648 heartbeater.cc:347] Connected to a master server at 127.20.80.254:37853
I20260929 10:52:36.880622 2648 heartbeater.cc:464] Registering TS with master...
I20260929 10:52:36.880810 2648 heartbeater.cc:511] Master 127.20.80.254:37853 requested a full tablet report, sending...
I20260929 10:52:36.881207 2185 ts_manager.cc:194] Registered new tserver with Master: 640879e300bf411c96666407655e9985 (127.20.80.195:39199)
I20260929 10:52:36.881593 2185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.195@KRBTEST.COM'} at 127.20.80.195:52685
I20260929 10:52:36.887590 20803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:52:36.894955 2225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:36.889046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35098 (local address 127.20.80.254:37853)
0929 10:52:36.889111 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:36.889113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:36.889281 (+ 168us) server_negotiation.cc:407] Connection header received
0929 10:52:36.889321 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:36.889321 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:36.889331 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:36.889392 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:36.889911 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.890478 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.891188 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.891367 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.891420 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:36.892539 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:36.892540 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:36.892541 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:36.892547 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:36.893927 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:36.894245 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:36.894246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:36.894246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:36.894300 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:36.894465 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:36.894466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:36.894466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:36.894603 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:36.894676 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:36.894763 (+ 87us) server_negotiation.cc:299] Negotiation successful
0929 10:52:36.894838 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 10:52:36.896624 2185 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35098:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:52:36.898519 2185 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:52:36.905448 2661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:36.901169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:43659 (local address 127.0.0.1:46922)
0929 10:52:36.901659 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:36.901670 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:36.901719 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:36.902176 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:36.902176 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:36.902179 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:52:36.902207 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:36.902208 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.903315 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.903316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:36.904207 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:36.904209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.905135 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.905137 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:36.905220 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:36.905236 (+ 16us) client_negotiation.cc:769] Sending connection context
0929 10:52:36.905311 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 10:52:36.905366 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":78,"threads_started":1}
I20260929 10:52:36.905568 2660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:36.900936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:39199 (local address 127.0.0.1:45166)
0929 10:52:36.901349 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:36.901361 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:36.901419 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:36.901990 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:36.901991 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:36.901993 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:52:36.902036 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:36.902037 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.902960 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.902962 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:36.904362 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:36.904366 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.905318 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.905319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:36.905384 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:36.905389 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 10:52:36.905452 (+ 63us) client_negotiation.cc:240] Negotiation successful
0929 10:52:36.905496 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"mutex_wait_us":54,"thread_start_us":99,"threads_started":1}
I20260929 10:52:36.905580 2662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:36.901180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45166 (local address 127.20.80.195:39199)
0929 10:52:36.901844 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:36.901848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:36.901857 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:52:36.901892 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:36.901894 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:36.901907 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:36.901979 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:52:36.902111 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.902842 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.904450 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.905225 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.905317 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:36.905374 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:36.905467 (+ 93us) server_negotiation.cc:299] Negotiation successful
0929 10:52:36.905505 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":81,"threads_started":1}
I20260929 10:52:36.905786 2664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:36.901403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46922 (local address 127.20.80.193:43659)
0929 10:52:36.901595 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:36.901599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:36.902048 (+ 449us) server_negotiation.cc:407] Connection header received
0929 10:52:36.902090 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:36.902093 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:36.902107 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:36.902177 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:52:36.902255 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.903211 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.904292 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.905050 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.905126 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:36.905193 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:36.905651 (+ 458us) server_negotiation.cc:299] Negotiation successful
0929 10:52:36.905710 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":121,"threads_started":1}
I20260929 10:52:36.906517 2659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:36.900936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:34205 (local address 127.0.0.1:43948)
0929 10:52:36.901087 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:36.901131 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:36.901201 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:36.901562 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:36.901566 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:36.901569 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:52:36.901663 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:36.901665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.903002 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.903003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:36.904363 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:36.904366 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.906261 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.906262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:36.906321 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:36.906325 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:52:36.906385 (+ 60us) client_negotiation.cc:240] Negotiation successful
0929 10:52:36.906439 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":62,"thread_start_us":89,"threads_started":1}
I20260929 10:52:36.906546 2663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:36.901175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43948 (local address 127.20.80.194:34205)
0929 10:52:36.901385 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:36.901390 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:36.901400 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:52:36.901444 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:36.901445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:36.901458 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:36.901558 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:52:36.901761 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.902916 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.905057 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.906113 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.906179 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:36.906228 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:36.906416 (+ 188us) server_negotiation.cc:299] Negotiation successful
0929 10:52:36.906461 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":99,"threads_started":1}
I20260929 10:52:36.906488 2311 tablet_service.cc:1467] Processing CreateTablet for tablet 958291a41a1843198eb7184dd8f6b54b (DEFAULT_TABLE table=test-table [id=f5b68a5d3c5b46b7bf6d4b3914d3936e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:52:36.906498 2580 tablet_service.cc:1467] Processing CreateTablet for tablet 958291a41a1843198eb7184dd8f6b54b (DEFAULT_TABLE table=test-table [id=f5b68a5d3c5b46b7bf6d4b3914d3936e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:52:36.906848 2580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 958291a41a1843198eb7184dd8f6b54b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:52:36.906857 2311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 958291a41a1843198eb7184dd8f6b54b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:52:36.907256 2446 tablet_service.cc:1467] Processing CreateTablet for tablet 958291a41a1843198eb7184dd8f6b54b (DEFAULT_TABLE table=test-table [id=f5b68a5d3c5b46b7bf6d4b3914d3936e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:52:36.907557 2446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 958291a41a1843198eb7184dd8f6b54b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:52:36.909578 2665 tablet_bootstrap.cc:492] T 958291a41a1843198eb7184dd8f6b54b P 640879e300bf411c96666407655e9985: Bootstrap starting.
I20260929 10:52:36.910096 2666 tablet_bootstrap.cc:492] T 958291a41a1843198eb7184dd8f6b54b P 6b59f952e57f4451904f3534100a4435: Bootstrap starting.
I20260929 10:52:36.910282 2665 tablet_bootstrap.cc:654] T 958291a41a1843198eb7184dd8f6b54b P 640879e300bf411c96666407655e9985: Neither blocks nor log segments found. Creating new log.
I20260929 10:52:36.910720 2667 tablet_bootstrap.cc:492] T 958291a41a1843198eb7184dd8f6b54b P b03c7542574e48749b01c123a1a93661: Bootstrap starting.
I20260929 10:52:36.910772 2666 tablet_bootstrap.cc:654] T 958291a41a1843198eb7184dd8f6b54b P 6b59f952e57f4451904f3534100a4435: Neither blocks nor log segments found. Creating new log.
I20260929 10:52:36.911010 2665 log.cc:824] T 958291a41a1843198eb7184dd8f6b54b P 640879e300bf411c96666407655e9985: Log is configured to *not* fsync() on all Append() calls
I20260929 10:52:36.911375 2667 tablet_bootstrap.cc:654] T 958291a41a1843198eb7184dd8f6b54b P b03c7542574e48749b01c123a1a93661: Neither blocks nor log segments found. Creating new log.
I20260929 10:52:36.911376 2666 log.cc:824] T 958291a41a1843198eb7184dd8f6b54b P 6b59f952e57f4451904f3534100a4435: Log is configured to *not* fsync() on all Append() calls
I20260929 10:52:36.911928 2665 tablet_bootstrap.cc:492] T 958291a41a1843198eb7184dd8f6b54b P 640879e300bf411c96666407655e9985: No bootstrap required, opened a new log
I20260929 10:52:36.911934 2667 log.cc:824] T 958291a41a1843198eb7184dd8f6b54b P b03c7542574e48749b01c123a1a93661: Log is configured to *not* fsync() on all Append() calls
I20260929 10:52:36.911983 2665 ts_tablet_manager.cc:1397] T 958291a41a1843198eb7184dd8f6b54b P 640879e300bf411c96666407655e9985: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:52:36.912292 2666 tablet_bootstrap.cc:492] T 958291a41a1843198eb7184dd8f6b54b P 6b59f952e57f4451904f3534100a4435: No bootstrap required, opened a new log
I20260929 10:52:36.912374 2666 ts_tablet_manager.cc:1397] T 958291a41a1843198eb7184dd8f6b54b P 6b59f952e57f4451904f3534100a4435: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:52:36.912750 2667 tablet_bootstrap.cc:492] T 958291a41a1843198eb7184dd8f6b54b P b03c7542574e48749b01c123a1a93661: No bootstrap required, opened a new log
I20260929 10:52:36.912834 2667 ts_tablet_manager.cc:1397] T 958291a41a1843198eb7184dd8f6b54b P b03c7542574e48749b01c123a1a93661: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:52:36.913758 2665 raft_consensus.cc:348] T 958291a41a1843198eb7184dd8f6b54b P 640879e300bf411c96666407655e9985 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b03c7542574e48749b01c123a1a93661" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34205 } } peers { permanent_uuid: "640879e300bf411c96666407655e9985" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39199 } } peers { permanent_uuid: "6b59f952e57f4451904f3534100a4435" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 43659 } }
I20260929 10:52:36.913924 2665 raft_consensus.cc:374] T 958291a41a1843198eb7184dd8f6b54b P 640879e300bf411c96666407655e9985 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:52:36.913968 2665 raft_consensus.cc:729] T 958291a41a1843198eb7184dd8f6b54b P 640879e300bf411c96666407655e9985 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 640879e300bf411c96666407655e9985, State: Initialized, Role: FOLLOWER
I20260929 10:52:36.914108 2665 consensus_queue.cc:260] T 958291a41a1843198eb7184dd8f6b54b P 640879e300bf411c96666407655e9985 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b03c7542574e48749b01c123a1a93661" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34205 } } peers { permanent_uuid: "640879e300bf411c96666407655e9985" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39199 } } peers { permanent_uuid: "6b59f952e57f4451904f3534100a4435" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 43659 } }
I20260929 10:52:36.914381 2648 heartbeater.cc:503] Master 127.20.80.254:37853 was elected leader, sending a full tablet report...
I20260929 10:52:36.914325 2666 raft_consensus.cc:348] T 958291a41a1843198eb7184dd8f6b54b P 6b59f952e57f4451904f3534100a4435 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b03c7542574e48749b01c123a1a93661" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34205 } } peers { permanent_uuid: "640879e300bf411c96666407655e9985" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39199 } } peers { permanent_uuid: "6b59f952e57f4451904f3534100a4435" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 43659 } }
I20260929 10:52:36.914496 2666 raft_consensus.cc:374] T 958291a41a1843198eb7184dd8f6b54b P 6b59f952e57f4451904f3534100a4435 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:52:36.914541 2666 raft_consensus.cc:729] T 958291a41a1843198eb7184dd8f6b54b P 6b59f952e57f4451904f3534100a4435 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b59f952e57f4451904f3534100a4435, State: Initialized, Role: FOLLOWER
I20260929 10:52:36.914381 2665 ts_tablet_manager.cc:1428] T 958291a41a1843198eb7184dd8f6b54b P 640879e300bf411c96666407655e9985: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 10:52:36.914654 2666 consensus_queue.cc:260] T 958291a41a1843198eb7184dd8f6b54b P 6b59f952e57f4451904f3534100a4435 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b03c7542574e48749b01c123a1a93661" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34205 } } peers { permanent_uuid: "640879e300bf411c96666407655e9985" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39199 } } peers { permanent_uuid: "6b59f952e57f4451904f3534100a4435" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 43659 } }
I20260929 10:52:36.914772 2667 raft_consensus.cc:348] T 958291a41a1843198eb7184dd8f6b54b P b03c7542574e48749b01c123a1a93661 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b03c7542574e48749b01c123a1a93661" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34205 } } peers { permanent_uuid: "640879e300bf411c96666407655e9985" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39199 } } peers { permanent_uuid: "6b59f952e57f4451904f3534100a4435" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 43659 } }
I20260929 10:52:36.914908 2666 ts_tablet_manager.cc:1428] T 958291a41a1843198eb7184dd8f6b54b P 6b59f952e57f4451904f3534100a4435: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260929 10:52:36.914935 2667 raft_consensus.cc:374] T 958291a41a1843198eb7184dd8f6b54b P b03c7542574e48749b01c123a1a93661 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:52:36.915010 2377 heartbeater.cc:503] Master 127.20.80.254:37853 was elected leader, sending a full tablet report...
I20260929 10:52:36.914971 2667 raft_consensus.cc:729] T 958291a41a1843198eb7184dd8f6b54b P b03c7542574e48749b01c123a1a93661 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b03c7542574e48749b01c123a1a93661, State: Initialized, Role: FOLLOWER
I20260929 10:52:36.915405 2667 consensus_queue.cc:260] T 958291a41a1843198eb7184dd8f6b54b P b03c7542574e48749b01c123a1a93661 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b03c7542574e48749b01c123a1a93661" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34205 } } peers { permanent_uuid: "640879e300bf411c96666407655e9985" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39199 } } peers { permanent_uuid: "6b59f952e57f4451904f3534100a4435" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 43659 } }
I20260929 10:52:36.915680 2667 ts_tablet_manager.cc:1428] T 958291a41a1843198eb7184dd8f6b54b P b03c7542574e48749b01c123a1a93661: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 10:52:36.915683 2512 heartbeater.cc:503] Master 127.20.80.254:37853 was elected leader, sending a full tablet report...
W20260929 10:52:36.959201 2513 tablet.cc:2369] T 958291a41a1843198eb7184dd8f6b54b P b03c7542574e48749b01c123a1a93661: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:52:36.986828 2673 raft_consensus.cc:482] T 958291a41a1843198eb7184dd8f6b54b P b03c7542574e48749b01c123a1a93661 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:52:36.987023 2673 raft_consensus.cc:504] T 958291a41a1843198eb7184dd8f6b54b P b03c7542574e48749b01c123a1a93661 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b03c7542574e48749b01c123a1a93661" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34205 } } peers { permanent_uuid: "640879e300bf411c96666407655e9985" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39199 } } peers { permanent_uuid: "6b59f952e57f4451904f3534100a4435" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 43659 } }
I20260929 10:52:36.987421 2673 leader_election.cc:288] T 958291a41a1843198eb7184dd8f6b54b P b03c7542574e48749b01c123a1a93661 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 640879e300bf411c96666407655e9985 (127.20.80.195:39199), 6b59f952e57f4451904f3534100a4435 (127.20.80.193:43659)
I20260929 10:52:36.990621 2662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:36.987724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:35599 (local address 127.20.80.195:39199)
0929 10:52:36.987772 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:36.987774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:36.987797 (+ 23us) server_negotiation.cc:407] Connection header received
0929 10:52:36.987934 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:36.987936 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:36.987947 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:36.988031 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:52:36.988221 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.988707 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.989816 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.990259 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.990351 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:36.990359 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:36.990515 (+ 156us) server_negotiation.cc:299] Negotiation successful
0929 10:52:36.990552 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 10:52:36.990621 2514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:36.987664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:39199 (local address 127.20.80.194:35599)
0929 10:52:36.987747 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:36.987761 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:36.987815 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:36.988047 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:36.988048 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:36.988050 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:52:36.988133 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:36.988135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.988814 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.988814 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:36.989725 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:36.989727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.990348 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.990349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:36.990433 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:36.990436 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:52:36.990508 (+ 72us) client_negotiation.cc:240] Negotiation successful
0929 10:52:36.990550 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260929 10:52:36.990779 2674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:36.987664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:43659 (local address 127.20.80.194:53671)
0929 10:52:36.987976 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:36.987986 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:36.988044 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:36.988204 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:36.988205 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:36.988206 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:52:36.988278 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:36.988279 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.989010 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.989011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:36.989813 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:36.989816 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.990521 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.990523 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:36.990593 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:36.990596 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:52:36.990662 (+ 66us) client_negotiation.cc:240] Negotiation successful
0929 10:52:36.990699 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":93,"thread_start_us":104,"threads_started":1}
I20260929 10:52:36.990768 2664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:36.987822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:53671 (local address 127.20.80.193:43659)
0929 10:52:36.987863 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:36.987865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:36.988047 (+ 182us) server_negotiation.cc:407] Connection header received
0929 10:52:36.988108 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:36.988109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:36.988118 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:36.988188 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:52:36.988357 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.988906 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.989985 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:36.990424 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:36.990511 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:36.990519 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:36.990668 (+ 149us) server_negotiation.cc:299] Negotiation successful
0929 10:52:36.990705 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 10:52:36.990994 2600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "958291a41a1843198eb7184dd8f6b54b" candidate_uuid: "b03c7542574e48749b01c123a1a93661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "640879e300bf411c96666407655e9985" is_pre_election: true
I20260929 10:52:36.991135 2331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "958291a41a1843198eb7184dd8f6b54b" candidate_uuid: "b03c7542574e48749b01c123a1a93661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b59f952e57f4451904f3534100a4435" is_pre_election: true
I20260929 10:52:36.991171 2600 raft_consensus.cc:2454] T 958291a41a1843198eb7184dd8f6b54b P 640879e300bf411c96666407655e9985 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b03c7542574e48749b01c123a1a93661 in term 0.
I20260929 10:52:36.991272 2331 raft_consensus.cc:2454] T 958291a41a1843198eb7184dd8f6b54b P 6b59f952e57f4451904f3534100a4435 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b03c7542574e48749b01c123a1a93661 in term 0.
I20260929 10:52:36.991492 2400 leader_election.cc:302] T 958291a41a1843198eb7184dd8f6b54b P b03c7542574e48749b01c123a1a93661 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 640879e300bf411c96666407655e9985, b03c7542574e48749b01c123a1a93661; no voters:
I20260929 10:52:36.991676 2673 raft_consensus.cc:2781] T 958291a41a1843198eb7184dd8f6b54b P b03c7542574e48749b01c123a1a93661 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:52:36.991715 2673 raft_consensus.cc:482] T 958291a41a1843198eb7184dd8f6b54b P b03c7542574e48749b01c123a1a93661 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:52:36.991731 2673 raft_consensus.cc:3037] T 958291a41a1843198eb7184dd8f6b54b P b03c7542574e48749b01c123a1a93661 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:52:36.992424 2673 raft_consensus.cc:504] T 958291a41a1843198eb7184dd8f6b54b P b03c7542574e48749b01c123a1a93661 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b03c7542574e48749b01c123a1a93661" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34205 } } peers { permanent_uuid: "640879e300bf411c96666407655e9985" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39199 } } peers { permanent_uuid: "6b59f952e57f4451904f3534100a4435" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 43659 } }
I20260929 10:52:36.992565 2673 leader_election.cc:288] T 958291a41a1843198eb7184dd8f6b54b P b03c7542574e48749b01c123a1a93661 [CANDIDATE]: Term 1 election: Requested vote from peers 640879e300bf411c96666407655e9985 (127.20.80.195:39199), 6b59f952e57f4451904f3534100a4435 (127.20.80.193:43659)
I20260929 10:52:36.992776 2331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "958291a41a1843198eb7184dd8f6b54b" candidate_uuid: "b03c7542574e48749b01c123a1a93661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b59f952e57f4451904f3534100a4435"
I20260929 10:52:36.992779 2600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "958291a41a1843198eb7184dd8f6b54b" candidate_uuid: "b03c7542574e48749b01c123a1a93661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "640879e300bf411c96666407655e9985"
I20260929 10:52:36.992859 2331 raft_consensus.cc:3037] T 958291a41a1843198eb7184dd8f6b54b P 6b59f952e57f4451904f3534100a4435 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:52:36.992880 2600 raft_consensus.cc:3037] T 958291a41a1843198eb7184dd8f6b54b P 640879e300bf411c96666407655e9985 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:52:36.993808 2331 raft_consensus.cc:2454] T 958291a41a1843198eb7184dd8f6b54b P 6b59f952e57f4451904f3534100a4435 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b03c7542574e48749b01c123a1a93661 in term 1.
I20260929 10:52:36.993826 2600 raft_consensus.cc:2454] T 958291a41a1843198eb7184dd8f6b54b P 640879e300bf411c96666407655e9985 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b03c7542574e48749b01c123a1a93661 in term 1.
I20260929 10:52:36.993983 2399 leader_election.cc:302] T 958291a41a1843198eb7184dd8f6b54b P b03c7542574e48749b01c123a1a93661 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b59f952e57f4451904f3534100a4435, b03c7542574e48749b01c123a1a93661; no voters:
I20260929 10:52:36.994083 2673 raft_consensus.cc:2781] T 958291a41a1843198eb7184dd8f6b54b P b03c7542574e48749b01c123a1a93661 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:52:36.994223 2673 raft_consensus.cc:686] T 958291a41a1843198eb7184dd8f6b54b P b03c7542574e48749b01c123a1a93661 [term 1 LEADER]: Becoming Leader. State: Replica: b03c7542574e48749b01c123a1a93661, State: Running, Role: LEADER
I20260929 10:52:36.994315 2673 consensus_queue.cc:237] T 958291a41a1843198eb7184dd8f6b54b P b03c7542574e48749b01c123a1a93661 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b03c7542574e48749b01c123a1a93661" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34205 } } peers { permanent_uuid: "640879e300bf411c96666407655e9985" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39199 } } peers { permanent_uuid: "6b59f952e57f4451904f3534100a4435" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 43659 } }
I20260929 10:52:36.995136 2185 catalog_manager.cc:5495] T 958291a41a1843198eb7184dd8f6b54b P b03c7542574e48749b01c123a1a93661 reported cstate change: term changed from 0 to 1, leader changed from <none> to b03c7542574e48749b01c123a1a93661 (127.20.80.194). New cstate: current_term: 1 leader_uuid: "b03c7542574e48749b01c123a1a93661" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b03c7542574e48749b01c123a1a93661" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "640879e300bf411c96666407655e9985" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b59f952e57f4451904f3534100a4435" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 43659 } health_report { overall_health: UNKNOWN } } }
I20260929 10:52:37.021397 2663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:37.018982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43952 (local address 127.20.80.194:34205)
0929 10:52:37.019066 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:37.019068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:37.019077 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:52:37.019118 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:37.019119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:37.019138 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:37.019203 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:52:37.019324 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.019817 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.020543 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.020769 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.020824 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:37.021038 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:52:37.021136 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:52:37.021215 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:37.021237 (+ 22us) server_negotiation.cc:299] Negotiation successful
0929 10:52:37.021256 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 10:52:37.023375 2331 raft_consensus.cc:1261] T 958291a41a1843198eb7184dd8f6b54b P 6b59f952e57f4451904f3534100a4435 [term 1 FOLLOWER]: Refusing update from remote peer b03c7542574e48749b01c123a1a93661: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:52:37.023373 2600 raft_consensus.cc:1261] T 958291a41a1843198eb7184dd8f6b54b P 640879e300bf411c96666407655e9985 [term 1 FOLLOWER]: Refusing update from remote peer b03c7542574e48749b01c123a1a93661: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:52:37.023823 2675 consensus_queue.cc:1035] T 958291a41a1843198eb7184dd8f6b54b P b03c7542574e48749b01c123a1a93661 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b59f952e57f4451904f3534100a4435" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:52:37.023977 2673 consensus_queue.cc:1035] T 958291a41a1843198eb7184dd8f6b54b P b03c7542574e48749b01c123a1a93661 [LEADER]: Connected to new peer: Peer: permanent_uuid: "640879e300bf411c96666407655e9985" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:52:37.026240 2678 mvcc.cc:205] Tried to move back new op lower bound from 7334621827164479488 to 7334621827049254912. Current Snapshot: MvccSnapshot[applied={T|T < 7334621827164479488}]
I20260929 10:52:37.028613 2185 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:52:37.028714 2185 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:52:37.029597 2185 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96: Sending DeleteTablet for 3 replicas of tablet 958291a41a1843198eb7184dd8f6b54b
I20260929 10:52:37.029964 2446 tablet_service.cc:1514] Processing DeleteTablet for tablet 958291a41a1843198eb7184dd8f6b54b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:52:37 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:43948
I20260929 10:52:37.029985 2580 tablet_service.cc:1514] Processing DeleteTablet for tablet 958291a41a1843198eb7184dd8f6b54b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:52:37 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:45166
I20260929 10:52:37.030244 2688 tablet_replica.cc:330] T 958291a41a1843198eb7184dd8f6b54b P b03c7542574e48749b01c123a1a93661: stopping tablet replica
I20260929 10:52:37.030221 2311 tablet_service.cc:1514] Processing DeleteTablet for tablet 958291a41a1843198eb7184dd8f6b54b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:52:37 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:46922
I20260929 10:52:37.030254 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 2248
I20260929 10:52:37.031148 2689 tablet_replica.cc:330] T 958291a41a1843198eb7184dd8f6b54b P 640879e300bf411c96666407655e9985: stopping tablet replica
I20260929 10:52:37.031263 2689 raft_consensus.cc:2229] T 958291a41a1843198eb7184dd8f6b54b P 640879e300bf411c96666407655e9985 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:52:37.031368 2689 raft_consensus.cc:2258] T 958291a41a1843198eb7184dd8f6b54b P 640879e300bf411c96666407655e9985 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:52:37.031526 2688 raft_consensus.cc:2229] T 958291a41a1843198eb7184dd8f6b54b P b03c7542574e48749b01c123a1a93661 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:52:37.031628 2688 raft_consensus.cc:2258] T 958291a41a1843198eb7184dd8f6b54b P b03c7542574e48749b01c123a1a93661 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:52:37.032053 2689 ts_tablet_manager.cc:1905] T 958291a41a1843198eb7184dd8f6b54b P 640879e300bf411c96666407655e9985: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:52:37.034119 2689 ts_tablet_manager.cc:1918] T 958291a41a1843198eb7184dd8f6b54b P 640879e300bf411c96666407655e9985: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:52:37.034168 2688 ts_tablet_manager.cc:1905] T 958291a41a1843198eb7184dd8f6b54b P b03c7542574e48749b01c123a1a93661: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:52:37.034191 2689 log.cc:1196] T 958291a41a1843198eb7184dd8f6b54b P 640879e300bf411c96666407655e9985: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/wal/wals/958291a41a1843198eb7184dd8f6b54b
I20260929 10:52:37.034498 2689 ts_tablet_manager.cc:1939] T 958291a41a1843198eb7184dd8f6b54b P 640879e300bf411c96666407655e9985: Deleting consensus metadata
I20260929 10:52:37.034889 2170 catalog_manager.cc:4841] TS 640879e300bf411c96666407655e9985 (127.20.80.195:39199): tablet 958291a41a1843198eb7184dd8f6b54b (table test-table [id=f5b68a5d3c5b46b7bf6d4b3914d3936e]) successfully deleted
I20260929 10:52:37.035784 2688 ts_tablet_manager.cc:1918] T 958291a41a1843198eb7184dd8f6b54b P b03c7542574e48749b01c123a1a93661: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:52:37.035843 2688 log.cc:1196] T 958291a41a1843198eb7184dd8f6b54b P b03c7542574e48749b01c123a1a93661: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/wal/wals/958291a41a1843198eb7184dd8f6b54b
I20260929 10:52:37.036111 2688 ts_tablet_manager.cc:1939] T 958291a41a1843198eb7184dd8f6b54b P b03c7542574e48749b01c123a1a93661: Deleting consensus metadata
I20260929 10:52:37.036675 2169 catalog_manager.cc:4841] TS b03c7542574e48749b01c123a1a93661 (127.20.80.194:34205): tablet 958291a41a1843198eb7184dd8f6b54b (table test-table [id=f5b68a5d3c5b46b7bf6d4b3914d3936e]) successfully deleted
W20260929 10:52:37.036988 2169 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.80.193:43659 (error 108)
I20260929 10:52:37.037643 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 2383
I20260929 10:52:37.037662 2659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:37.037505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:43659 (local address 127.0.0.1:46934)
0929 10:52:37.037572 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:37.037626 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.193:43659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260929 10:52:37.037776 2169 catalog_manager.cc:4584] TS 6b59f952e57f4451904f3534100a4435 (127.20.80.193:43659): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 958291a41a1843198eb7184dd8f6b54b: Network error: Client connection negotiation failed: client connection to 127.20.80.193:43659: connect: Connection refused (error 111)
I20260929 10:52:37.043650 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 2517
I20260929 10:52:37.048764 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 2153
I20260929 10:52:37.054184 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:37853
--webserver_interface=127.20.80.254
--webserver_port=33955
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:33539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:37853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:52:37.135917 2691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:52:37.136121 2691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:52:37.136145 2691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:52:37.137362 2691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:52:37.137404 2691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:52:37.137418 2691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:52:37.137432 2691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:52:37.137480 2691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:52:37.138927 2691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:33539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:37853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:37853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=33955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:52:37.139178 2691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:52:37.139407 2691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:52:37.149142 2700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:37.149296 2691 server_base.cc:1034] running on GCE node
W20260929 10:52:37.149384 2697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:37.149183 2698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:37.149710 2691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:52:37.150038 2691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:52:37.151189 2691 hybrid_clock.cc:648] HybridClock initialized: now 1790679157151152 us; error 53 us; skew 500 ppm
Sep 29 10:52:37 dist-test-slave-vqch krb5kdc[2140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679157, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:52:37.158972 2691 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:52:37.159448 2691 webserver.cc:460] Webserver started at http://127.20.80.254:33955/ using document root <none> and password file <none>
I20260929 10:52:37.159704 2691 fs_manager.cc:359] Metadata directory not provided
I20260929 10:52:37.159762 2691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:52:37.161234 2691 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:52:37.161984 2707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:37.162189 2691 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:52:37.162297 2691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "1ab3cc1989514145aad5dfa61e847b96"
format_stamp: "Formatted at 2026-09-29 10:52:36 on dist-test-slave-vqch"
server_key: "61726378975ad41bd47ed6428652b572"
server_key_iv: "a271787484b42854562d28a652b506e3"
server_key_version: "encryptionkey@0"
I20260929 10:52:37.162698 2691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:52:37.174908 2691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:52:37.176949 2691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:52:37.177147 2691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:52:37.181115 2691 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:37853
I20260929 10:52:37.181217 2760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:37853 every 8 connection(s)
I20260929 10:52:37.181578 2691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:52:37.183233 2761 sys_catalog.cc:263] Verifying existing consensus state
I20260929 10:52:37.183830 2761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96: Bootstrap starting.
I20260929 10:52:37.189067 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 2691
I20260929 10:52:37.189352 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.193:43659
--local_ip_for_outbound_sockets=127.20.80.193
--tserver_master_addrs=127.20.80.254:37853
--webserver_port=46475
--webserver_interface=127.20.80.193
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:33539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:52:37.192525 2761 log.cc:824] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96: Log is configured to *not* fsync() on all Append() calls
I20260929 10:52:37.194633 2761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 10:52:37.194962 2761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96: Bootstrap complete.
I20260929 10:52:37.197327 2761 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab3cc1989514145aad5dfa61e847b96" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37853 } }
I20260929 10:52:37.197551 2761 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ab3cc1989514145aad5dfa61e847b96, State: Initialized, Role: FOLLOWER
I20260929 10:52:37.197723 2761 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab3cc1989514145aad5dfa61e847b96" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37853 } }
I20260929 10:52:37.197783 2761 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:52:37.197835 2761 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:52:37.197885 2761 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [term 1 FOLLOWER]: Advancing to term 2
I20260929 10:52:37.199008 2761 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab3cc1989514145aad5dfa61e847b96" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37853 } }
I20260929 10:52:37.199095 2761 leader_election.cc:302] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ab3cc1989514145aad5dfa61e847b96; no voters:
I20260929 10:52:37.199306 2761 leader_election.cc:288] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 10:52:37.199339 2765 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 10:52:37.199443 2765 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [term 2 LEADER]: Becoming Leader. State: Replica: 1ab3cc1989514145aad5dfa61e847b96, State: Running, Role: LEADER
I20260929 10:52:37.199574 2761 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:52:37.199590 2765 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab3cc1989514145aad5dfa61e847b96" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37853 } }
I20260929 10:52:37.199983 2767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ab3cc1989514145aad5dfa61e847b96. Latest consensus state: current_term: 2 leader_uuid: "1ab3cc1989514145aad5dfa61e847b96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab3cc1989514145aad5dfa61e847b96" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37853 } } }
I20260929 10:52:37.200069 2767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [sys.catalog]: This master's current role is: LEADER
I20260929 10:52:37.200394 2771 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:52:37.200464 2766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1ab3cc1989514145aad5dfa61e847b96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab3cc1989514145aad5dfa61e847b96" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37853 } } }
I20260929 10:52:37.200600 2766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [sys.catalog]: This master's current role is: LEADER
I20260929 10:52:37.201359 2771 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:52:37.201501 2771 catalog_manager.cc:1259] Loaded cluster ID: 52e1580f6b814e2d8b2a82cec0f7b7e5
I20260929 10:52:37.201722 2771 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:52:37.201963 2771 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260929 10:52:37.206627 2771 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260929 10:52:37.209213 2771 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:52:37.209438 2771 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96: Loaded TSK: 0
I20260929 10:52:37.209672 2771 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 10:52:37.273494 2763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:52:37.273762 2763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:52:37.273789 2763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:52:37.274973 2763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:52:37.275013 2763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:52:37.275039 2763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.193
I20260929 10:52:37.276520 2763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:33539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.193:43659
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.80.193
--webserver_port=46475
--tserver_master_addrs=127.20.80.254:37853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.193
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:52:37.276762 2763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:52:37.277061 2763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:52:37.282107 2788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:37.282078 2791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:37.282153 2789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:37.282404 2763 server_base.cc:1034] running on GCE node
I20260929 10:52:37.282619 2763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:52:37.282900 2763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:52:37.284082 2763 hybrid_clock.cc:648] HybridClock initialized: now 1790679157284060 us; error 36 us; skew 500 ppm
Sep 29 10:52:37 dist-test-slave-vqch krb5kdc[2140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679157, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:52:37.286700 2763 init.cc:377] Logged in from keytab as kudu/127.20.80.193@KRBTEST.COM (short username kudu)
I20260929 10:52:37.287230 2763 webserver.cc:460] Webserver started at http://127.20.80.193:46475/ using document root <none> and password file <none>
I20260929 10:52:37.287528 2763 fs_manager.cc:359] Metadata directory not provided
I20260929 10:52:37.287618 2763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:52:37.289196 2763 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:52:37.289927 2799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:37.290072 2763 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:52:37.290156 2763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/wal
uuid: "6b59f952e57f4451904f3534100a4435"
format_stamp: "Formatted at 2026-09-29 10:52:36 on dist-test-slave-vqch"
server_key: "6bcb6adfd5e3334122d7e4e26f49e70d"
server_key_iv: "ca704924dffe88c3936aaf7d35064945"
server_key_version: "encryptionkey@0"
I20260929 10:52:37.290530 2763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:52:37.306828 2763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:52:37.309154 2763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:52:37.309303 2763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:52:37.309584 2763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:52:37.310097 2806 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 10:52:37.311594 2763 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 10:52:37.311646 2763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260929 10:52:37.311681 2763 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 10:52:37.312359 2763 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 10:52:37.312433 2763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260929 10:52:37.312471 2806 tablet_bootstrap.cc:492] T 958291a41a1843198eb7184dd8f6b54b P 6b59f952e57f4451904f3534100a4435: Bootstrap starting.
I20260929 10:52:37.320276 2763 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.193:43659
I20260929 10:52:37.320474 2913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.193:43659 every 8 connection(s)
I20260929 10:52:37.320827 2763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
I20260929 10:52:37.324504 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 2763
I20260929 10:52:37.324611 2806 log.cc:824] T 958291a41a1843198eb7184dd8f6b54b P 6b59f952e57f4451904f3534100a4435: Log is configured to *not* fsync() on all Append() calls
I20260929 10:52:37.324774 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.194:34205
--local_ip_for_outbound_sockets=127.20.80.194
--tserver_master_addrs=127.20.80.254:37853
--webserver_port=35347
--webserver_interface=127.20.80.194
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:33539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:52:37.327126 2806 tablet_bootstrap.cc:492] T 958291a41a1843198eb7184dd8f6b54b P 6b59f952e57f4451904f3534100a4435: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 10:52:37.327471 2806 tablet_bootstrap.cc:492] T 958291a41a1843198eb7184dd8f6b54b P 6b59f952e57f4451904f3534100a4435: Bootstrap complete.
I20260929 10:52:37.328662 2806 ts_tablet_manager.cc:1397] T 958291a41a1843198eb7184dd8f6b54b P 6b59f952e57f4451904f3534100a4435: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.009s
Sep 29 10:52:37 dist-test-slave-vqch krb5kdc[2140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679157, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:52:37.329841 2806 raft_consensus.cc:348] T 958291a41a1843198eb7184dd8f6b54b P 6b59f952e57f4451904f3534100a4435 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b03c7542574e48749b01c123a1a93661" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34205 } } peers { permanent_uuid: "640879e300bf411c96666407655e9985" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39199 } } peers { permanent_uuid: "6b59f952e57f4451904f3534100a4435" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 43659 } }
I20260929 10:52:37.330219 2806 raft_consensus.cc:729] T 958291a41a1843198eb7184dd8f6b54b P 6b59f952e57f4451904f3534100a4435 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b59f952e57f4451904f3534100a4435, State: Initialized, Role: FOLLOWER
I20260929 10:52:37.330364 2806 consensus_queue.cc:260] T 958291a41a1843198eb7184dd8f6b54b P 6b59f952e57f4451904f3534100a4435 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b03c7542574e48749b01c123a1a93661" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34205 } } peers { permanent_uuid: "640879e300bf411c96666407655e9985" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39199 } } peers { permanent_uuid: "6b59f952e57f4451904f3534100a4435" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 43659 } }
I20260929 10:52:37.330683 2806 ts_tablet_manager.cc:1428] T 958291a41a1843198eb7184dd8f6b54b P 6b59f952e57f4451904f3534100a4435: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
I20260929 10:52:37.334149 2916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:37.321596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.193:51177)
0929 10:52:37.321817 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:37.321866 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:37.322450 (+ 584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:37.323003 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:37.323007 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:37.323369 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:52:37.323620 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:37.323626 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.325348 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.325349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:37.326600 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:37.326603 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.327424 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.327425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:37.327578 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:37.328183 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:52:37.328194 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:52:37.330027 (+ 1833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:52:37.332340 (+ 2313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:37.332347 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:37.332349 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:37.332625 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:37.332971 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:37.332971 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:37.332972 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:37.333122 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:37.333633 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:52:37.333642 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:52:37.333803 (+ 161us) client_negotiation.cc:769] Sending connection context
0929 10:52:37.333890 (+ 87us) client_negotiation.cc:240] Negotiation successful
0929 10:52:37.334011 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":123,"threads_started":1}
I20260929 10:52:37.334412 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:37.321739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:51177 (local address 127.20.80.254:37853)
0929 10:52:37.321922 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:37.321932 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:37.321988 (+ 56us) server_negotiation.cc:407] Connection header received
0929 10:52:37.322703 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:37.322711 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:37.322793 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:37.322974 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:37.323764 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.325217 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.326697 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.327302 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.327403 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:37.330146 (+ 2743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:37.330150 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:37.330150 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:37.330162 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:37.332235 (+ 2073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:37.332751 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:37.332755 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:37.332755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:37.332885 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:37.333199 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:37.333201 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:37.333201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:37.333532 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:37.333630 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:37.333997 (+ 367us) server_negotiation.cc:299] Negotiation successful
0929 10:52:37.334161 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":74,"threads_started":1}
I20260929 10:52:37.335112 2914 heartbeater.cc:347] Connected to a master server at 127.20.80.254:37853
I20260929 10:52:37.335234 2914 heartbeater.cc:464] Registering TS with master...
I20260929 10:52:37.335404 2914 heartbeater.cc:511] Master 127.20.80.254:37853 requested a full tablet report, sending...
I20260929 10:52:37.336131 2725 ts_manager.cc:194] Registered new tserver with Master: 6b59f952e57f4451904f3534100a4435 (127.20.80.193:43659)
I20260929 10:52:37.337383 2725 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.193@KRBTEST.COM'} at 127.20.80.193:51177
I20260929 10:52:37.338335 2914 heartbeater.cc:503] Master 127.20.80.254:37853 was elected leader, sending a full tablet report...
Sep 29 10:52:37 dist-test-slave-vqch krb5kdc[2140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679157, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for kudu/127.20.80.193@KRBTEST.COM
I20260929 10:52:37.346170 2921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:37.336840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:43659 (local address 127.0.0.1:46936)
0929 10:52:37.337187 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:37.337224 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:37.337301 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:37.337565 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:37.337572 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:37.337955 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:52:37.338022 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:37.338025 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.339123 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.339124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:37.340208 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:37.340211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.340727 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.340728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:37.340824 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:37.341344 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:52:37.341350 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:52:37.342896 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:52:37.345060 (+ 2164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:37.345068 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:37.345070 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:37.345273 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:37.345509 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:37.345514 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:37.345514 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:37.345580 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:37.345892 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:52:37.345898 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:52:37.345941 (+ 43us) client_negotiation.cc:769] Sending connection context
0929 10:52:37.346007 (+ 66us) client_negotiation.cc:240] Negotiation successful
0929 10:52:37.346101 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":106,"threads_started":1}
I20260929 10:52:37.346256 2922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:37.336932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46936 (local address 127.20.80.193:43659)
0929 10:52:37.337105 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:37.337109 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:37.337310 (+ 201us) server_negotiation.cc:407] Connection header received
0929 10:52:37.337382 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:37.337385 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:37.337406 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:37.337485 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:37.338119 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.339047 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.340320 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.340609 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.340705 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:37.343016 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:37.343025 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:37.343025 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:37.343032 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:37.344942 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:37.345363 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:37.345366 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:37.345366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:37.345435 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:37.345644 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:37.345645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:37.345645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:37.345821 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:37.345904 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:37.346063 (+ 159us) server_negotiation.cc:299] Negotiation successful
0929 10:52:37.346161 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":101,"threads_started":1}
I20260929 10:52:37.346679 2848 tablet_service.cc:1514] Processing DeleteTablet for tablet 958291a41a1843198eb7184dd8f6b54b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:52:37 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:46936
I20260929 10:52:37.347299 2923 tablet_replica.cc:330] stopping tablet replica
I20260929 10:52:37.347420 2923 raft_consensus.cc:2229] T 958291a41a1843198eb7184dd8f6b54b P 6b59f952e57f4451904f3534100a4435 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:52:37.347527 2923 raft_consensus.cc:2258] T 958291a41a1843198eb7184dd8f6b54b P 6b59f952e57f4451904f3534100a4435 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:52:37.347853 2923 ts_tablet_manager.cc:1905] T 958291a41a1843198eb7184dd8f6b54b P 6b59f952e57f4451904f3534100a4435: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:52:37.350168 2923 ts_tablet_manager.cc:1918] T 958291a41a1843198eb7184dd8f6b54b P 6b59f952e57f4451904f3534100a4435: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:52:37.350260 2923 log.cc:1196] T 958291a41a1843198eb7184dd8f6b54b P 6b59f952e57f4451904f3534100a4435: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/wal/wals/958291a41a1843198eb7184dd8f6b54b
I20260929 10:52:37.350490 2923 ts_tablet_manager.cc:1939] T 958291a41a1843198eb7184dd8f6b54b P 6b59f952e57f4451904f3534100a4435: Deleting consensus metadata
I20260929 10:52:37.351022 2708 catalog_manager.cc:4841] TS 6b59f952e57f4451904f3534100a4435 (127.20.80.193:43659): tablet 958291a41a1843198eb7184dd8f6b54b (table test-table [id=f5b68a5d3c5b46b7bf6d4b3914d3936e]) successfully deleted
W20260929 10:52:37.418387 2919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:52:37.418584 2919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:52:37.418607 2919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:52:37.419891 2919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:52:37.419938 2919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:52:37.420140 2919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.194
I20260929 10:52:37.421792 2919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:33539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.194:34205
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.80.194
--webserver_port=35347
--tserver_master_addrs=127.20.80.254:37853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.194
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:52:37.422042 2919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:52:37.422281 2919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:52:37.428506 2929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:37.428586 2931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:37.428898 2919 server_base.cc:1034] running on GCE node
W20260929 10:52:37.429421 2928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:37.429678 2919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:52:37.429980 2919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:52:37.431128 2919 hybrid_clock.cc:648] HybridClock initialized: now 1790679157431115 us; error 28 us; skew 500 ppm
Sep 29 10:52:37 dist-test-slave-vqch krb5kdc[2140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679157, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:52:37.434659 2919 init.cc:377] Logged in from keytab as kudu/127.20.80.194@KRBTEST.COM (short username kudu)
I20260929 10:52:37.435133 2919 webserver.cc:460] Webserver started at http://127.20.80.194:35347/ using document root <none> and password file <none>
I20260929 10:52:37.435318 2919 fs_manager.cc:359] Metadata directory not provided
I20260929 10:52:37.435369 2919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:52:37.436754 2919 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:52:37.437448 2939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:37.437606 2919 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:37.437703 2919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/wal
uuid: "b03c7542574e48749b01c123a1a93661"
format_stamp: "Formatted at 2026-09-29 10:52:36 on dist-test-slave-vqch"
server_key: "65b0caa4c3eea46eeb1479b88b780f00"
server_key_iv: "2683f4a25f1de177be466753e13f1ce5"
server_key_version: "encryptionkey@0"
I20260929 10:52:37.438028 2919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:52:37.454789 2919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:52:37.457136 2919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:52:37.457253 2919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:52:37.457439 2919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:52:37.457767 2919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:52:37.457801 2919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:37.457824 2919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:52:37.457877 2919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:37.463760 2919 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.194:34205
I20260929 10:52:37.463924 3052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.194:34205 every 8 connection(s)
I20260929 10:52:37.464201 2919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
I20260929 10:52:37.465417 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 2919
I20260929 10:52:37.465732 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.195:39199
--local_ip_for_outbound_sockets=127.20.80.195
--tserver_master_addrs=127.20.80.254:37853
--webserver_port=34129
--webserver_interface=127.20.80.195
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:33539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 10:52:37 dist-test-slave-vqch krb5kdc[2140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679157, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:52:37.475622 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:37.464951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:52675 (local address 127.20.80.254:37853)
0929 10:52:37.465022 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:37.465024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:37.465322 (+ 298us) server_negotiation.cc:407] Connection header received
0929 10:52:37.466081 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:37.466083 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:37.466094 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:37.466160 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:37.467484 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.468283 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.469881 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.470090 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.470151 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:37.472643 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:37.472645 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:37.472646 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:37.472654 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:37.474146 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:37.474581 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:37.474583 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:37.474583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:37.474642 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:37.474939 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:37.474940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:37.474941 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:37.475062 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:37.475138 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:37.475431 (+ 293us) server_negotiation.cc:299] Negotiation successful
0929 10:52:37.475527 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 10:52:37.475808 3055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:37.464916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.194:52675)
0929 10:52:37.465130 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:37.465165 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:37.465953 (+ 788us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:37.466233 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:37.466238 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:37.467063 (+ 825us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:52:37.467376 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:37.467383 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.468383 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.468384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:37.469786 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:37.469789 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.470189 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.470190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:37.470351 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:37.470891 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:52:37.470902 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:52:37.472536 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:52:37.474235 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:37.474243 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:37.474245 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:37.474482 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:37.474714 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:37.474715 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:37.474715 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:37.474834 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:37.475140 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:52:37.475148 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:52:37.475292 (+ 144us) client_negotiation.cc:769] Sending connection context
0929 10:52:37.475407 (+ 115us) client_negotiation.cc:240] Negotiation successful
0929 10:52:37.475570 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":131,"threads_started":1}
I20260929 10:52:37.476294 3053 heartbeater.cc:347] Connected to a master server at 127.20.80.254:37853
I20260929 10:52:37.476411 3053 heartbeater.cc:464] Registering TS with master...
I20260929 10:52:37.476627 3053 heartbeater.cc:511] Master 127.20.80.254:37853 requested a full tablet report, sending...
I20260929 10:52:37.477069 2725 ts_manager.cc:194] Registered new tserver with Master: b03c7542574e48749b01c123a1a93661 (127.20.80.194:34205)
I20260929 10:52:37.477473 2725 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.194@KRBTEST.COM'} at 127.20.80.194:52675
W20260929 10:52:37.553139 3056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:52:37.553329 3056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:52:37.553351 3056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:52:37.554526 3056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:52:37.554569 3056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:52:37.554594 3056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.195
I20260929 10:52:37.556063 3056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:33539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.195:39199
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.80.195
--webserver_port=34129
--tserver_master_addrs=127.20.80.254:37853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.195
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:52:37.556313 3056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:52:37.556541 3056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:52:37.561575 3062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:37.561609 3064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:37.561563 3061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:37.561854 3056 server_base.cc:1034] running on GCE node
I20260929 10:52:37.562089 3056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:52:37.562347 3056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:52:37.563517 3056 hybrid_clock.cc:648] HybridClock initialized: now 1790679157563513 us; error 37 us; skew 500 ppm
Sep 29 10:52:37 dist-test-slave-vqch krb5kdc[2140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679157, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:52:37.566243 3056 init.cc:377] Logged in from keytab as kudu/127.20.80.195@KRBTEST.COM (short username kudu)
I20260929 10:52:37.566715 3056 webserver.cc:460] Webserver started at http://127.20.80.195:34129/ using document root <none> and password file <none>
I20260929 10:52:37.566967 3056 fs_manager.cc:359] Metadata directory not provided
I20260929 10:52:37.567049 3056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:52:37.568738 3056 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:52:37.569496 3071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:37.569741 3056 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 10:52:37.569841 3056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/wal
uuid: "640879e300bf411c96666407655e9985"
format_stamp: "Formatted at 2026-09-29 10:52:36 on dist-test-slave-vqch"
server_key: "cc356bc2c12fcda8cb3e86049eba111e"
server_key_iv: "313662a96148ee583226440d8020aacd"
server_key_version: "encryptionkey@0"
I20260929 10:52:37.570294 3056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:52:37.578425 3056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:52:37.580420 3056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:52:37.580525 3056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:52:37.580694 3056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:52:37.581239 3056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:52:37.581314 3056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:37.581362 3056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:52:37.581387 3056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:37.587076 3056 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.195:39199
I20260929 10:52:37.587169 3185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.195:39199 every 8 connection(s)
I20260929 10:52:37.587558 3056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
I20260929 10:52:37.591537 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 3056
Sep 29 10:52:37 dist-test-slave-vqch krb5kdc[2140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679157, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:52:37.597627 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:37.588257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:55011 (local address 127.20.80.254:37853)
0929 10:52:37.588318 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:37.588320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:37.588561 (+ 241us) server_negotiation.cc:407] Connection header received
0929 10:52:37.589188 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:37.589189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:37.589200 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:37.589274 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:37.590173 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.590677 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.591680 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.592004 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.592081 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:37.594520 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:37.594522 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:37.594522 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:37.594531 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:37.595908 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:37.596509 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:37.596511 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:37.596511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:37.596576 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:37.596828 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:37.596829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:37.596829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:37.596947 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:37.597081 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:37.597361 (+ 280us) server_negotiation.cc:299] Negotiation successful
0929 10:52:37.597537 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 10:52:37.597764 3188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:37.588263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.195:55011)
0929 10:52:37.588414 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:37.588447 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:37.589054 (+ 607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:37.589332 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:37.589335 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:37.589882 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:52:37.590064 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:37.590069 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.590762 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.590762 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:37.591546 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:37.591548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.592152 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.592153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:37.592310 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:37.592829 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:52:37.592839 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:52:37.594424 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:52:37.596212 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:37.596219 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:37.596221 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:37.596428 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:37.596646 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:37.596647 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:37.596647 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:37.596753 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:37.597074 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:52:37.597081 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:52:37.597217 (+ 136us) client_negotiation.cc:769] Sending connection context
0929 10:52:37.597321 (+ 104us) client_negotiation.cc:240] Negotiation successful
0929 10:52:37.597489 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":75,"threads_started":1}
I20260929 10:52:37.598294 3186 heartbeater.cc:347] Connected to a master server at 127.20.80.254:37853
I20260929 10:52:37.598397 3186 heartbeater.cc:464] Registering TS with master...
I20260929 10:52:37.598608 3186 heartbeater.cc:511] Master 127.20.80.254:37853 requested a full tablet report, sending...
I20260929 10:52:37.598924 3190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:37.591965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35100 (local address 127.20.80.254:37853)
0929 10:52:37.592075 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:37.592076 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:37.592120 (+ 44us) server_negotiation.cc:407] Connection header received
0929 10:52:37.592195 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:37.592195 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:37.592205 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:37.592255 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:37.593077 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.593641 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.594574 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.594784 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.594836 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:37.595998 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:37.595999 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:37.595999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:37.596007 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:37.597580 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:37.598050 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:37.598051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:37.598051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:37.598117 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:37.598326 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:37.598327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:37.598327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:37.598476 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:37.598610 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:37.598716 (+ 106us) server_negotiation.cc:299] Negotiation successful
0929 10:52:37.598816 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":70,"threads_started":1}
I20260929 10:52:37.599068 2725 ts_manager.cc:194] Registered new tserver with Master: 640879e300bf411c96666407655e9985 (127.20.80.195:39199)
I20260929 10:52:37.599582 2725 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.195@KRBTEST.COM'} at 127.20.80.195:55011
I20260929 10:52:37.599645 20803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:52:37.602407 3190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:37.600001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35108 (local address 127.20.80.254:37853)
0929 10:52:37.600037 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:37.600039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:37.600156 (+ 117us) server_negotiation.cc:407] Connection header received
0929 10:52:37.600195 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:37.600195 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:37.600228 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:37.600307 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:52:37.600393 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.601082 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.601788 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.601979 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.602047 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:37.602170 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:52:37.602252 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:52:37.602312 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:37.602345 (+ 33us) server_negotiation.cc:299] Negotiation successful
0929 10:52:37.602363 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 10:52:37.602679 2725 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:35108:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:52:37.604337 2725 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:52:37.606662 2848 tablet_service.cc:1467] Processing CreateTablet for tablet 6625b40503604fffb0a8a9078bb24328 (DEFAULT_TABLE table=test-table [id=2235ef1a993342e3ae149af5662f0d9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:52:37.606899 2848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6625b40503604fffb0a8a9078bb24328. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:52:37.608426 2806 tablet_bootstrap.cc:492] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435: Bootstrap starting.
I20260929 10:52:37.608781 2806 tablet_bootstrap.cc:654] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435: Neither blocks nor log segments found. Creating new log.
I20260929 10:52:37.609452 2806 tablet_bootstrap.cc:492] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435: No bootstrap required, opened a new log
I20260929 10:52:37.609511 2806 ts_tablet_manager.cc:1397] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 10:52:37.609863 2806 raft_consensus.cc:348] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "640879e300bf411c96666407655e9985" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39199 } } peers { permanent_uuid: "b03c7542574e48749b01c123a1a93661" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34205 } } peers { permanent_uuid: "6b59f952e57f4451904f3534100a4435" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 43659 } }
I20260929 10:52:37.609974 2806 raft_consensus.cc:374] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:52:37.610028 2806 raft_consensus.cc:729] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b59f952e57f4451904f3534100a4435, State: Initialized, Role: FOLLOWER
I20260929 10:52:37.610080 2806 consensus_queue.cc:260] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "640879e300bf411c96666407655e9985" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39199 } } peers { permanent_uuid: "b03c7542574e48749b01c123a1a93661" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34205 } } peers { permanent_uuid: "6b59f952e57f4451904f3534100a4435" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 43659 } }
I20260929 10:52:37.610195 2806 ts_tablet_manager.cc:1428] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260929 10:52:37.610610 3192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:37.606232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:34205 (local address 127.0.0.1:43962)
0929 10:52:37.606535 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:37.606546 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:37.606604 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:37.606789 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:37.606790 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:37.606792 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:52:37.606843 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:37.606845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.608004 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.608005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:37.609393 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:37.609396 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.610346 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.610347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:37.610422 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:37.610442 (+ 20us) client_negotiation.cc:769] Sending connection context
0929 10:52:37.610498 (+ 56us) client_negotiation.cc:240] Negotiation successful
0929 10:52:37.610541 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":15,"thread_start_us":90,"threads_started":1}
I20260929 10:52:37.610646 3193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:37.606349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43962 (local address 127.20.80.194:34205)
0929 10:52:37.606529 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:37.606534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:37.606594 (+ 60us) server_negotiation.cc:407] Connection header received
0929 10:52:37.606668 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:37.606670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:37.606684 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:37.606756 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:52:37.606952 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.607894 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.609479 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.610267 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.610333 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:37.610392 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:37.610522 (+ 130us) server_negotiation.cc:299] Negotiation successful
0929 10:52:37.610565 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":97,"threads_started":1}
I20260929 10:52:37.611310 2987 tablet_service.cc:1467] Processing CreateTablet for tablet 6625b40503604fffb0a8a9078bb24328 (DEFAULT_TABLE table=test-table [id=2235ef1a993342e3ae149af5662f0d9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:52:37.611698 2987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6625b40503604fffb0a8a9078bb24328. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:52:37.612188 2921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:37.606232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:39199 (local address 127.0.0.1:45170)
0929 10:52:37.606294 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:37.606308 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:37.606360 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:37.608639 (+ 2279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:37.608639 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:37.608641 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:52:37.608676 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:37.608677 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.609941 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.609942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:37.610929 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:37.610932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.611927 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.611929 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:37.612003 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:37.612008 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 10:52:37.612072 (+ 64us) client_negotiation.cc:240] Negotiation successful
0929 10:52:37.612120 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260929 10:52:37.612226 3194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:37.606398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45170 (local address 127.20.80.195:39199)
0929 10:52:37.608466 (+ 2068us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:37.608471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:37.608482 (+ 11us) server_negotiation.cc:407] Connection header received
0929 10:52:37.608515 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:37.608517 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:37.608532 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:37.608736 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:52:37.608791 (+ 55us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.609827 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.611047 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.611818 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.611925 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:37.611982 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:37.612087 (+ 105us) server_negotiation.cc:299] Negotiation successful
0929 10:52:37.612126 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2036,"thread_start_us":94,"threads_started":1}
I20260929 10:52:37.612928 3120 tablet_service.cc:1467] Processing CreateTablet for tablet 6625b40503604fffb0a8a9078bb24328 (DEFAULT_TABLE table=test-table [id=2235ef1a993342e3ae149af5662f0d9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:52:37.613279 3120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6625b40503604fffb0a8a9078bb24328. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:52:37.615388 3196 tablet_bootstrap.cc:492] T 6625b40503604fffb0a8a9078bb24328 P b03c7542574e48749b01c123a1a93661: Bootstrap starting.
I20260929 10:52:37.616128 3196 tablet_bootstrap.cc:654] T 6625b40503604fffb0a8a9078bb24328 P b03c7542574e48749b01c123a1a93661: Neither blocks nor log segments found. Creating new log.
I20260929 10:52:37.616570 3197 tablet_bootstrap.cc:492] T 6625b40503604fffb0a8a9078bb24328 P 640879e300bf411c96666407655e9985: Bootstrap starting.
I20260929 10:52:37.616772 3196 log.cc:824] T 6625b40503604fffb0a8a9078bb24328 P b03c7542574e48749b01c123a1a93661: Log is configured to *not* fsync() on all Append() calls
I20260929 10:52:37.617234 3197 tablet_bootstrap.cc:654] T 6625b40503604fffb0a8a9078bb24328 P 640879e300bf411c96666407655e9985: Neither blocks nor log segments found. Creating new log.
I20260929 10:52:37.617632 3196 tablet_bootstrap.cc:492] T 6625b40503604fffb0a8a9078bb24328 P b03c7542574e48749b01c123a1a93661: No bootstrap required, opened a new log
I20260929 10:52:37.617733 3196 ts_tablet_manager.cc:1397] T 6625b40503604fffb0a8a9078bb24328 P b03c7542574e48749b01c123a1a93661: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:52:37.617887 3197 log.cc:824] T 6625b40503604fffb0a8a9078bb24328 P 640879e300bf411c96666407655e9985: Log is configured to *not* fsync() on all Append() calls
I20260929 10:52:37.618624 3197 tablet_bootstrap.cc:492] T 6625b40503604fffb0a8a9078bb24328 P 640879e300bf411c96666407655e9985: No bootstrap required, opened a new log
I20260929 10:52:37.618701 3197 ts_tablet_manager.cc:1397] T 6625b40503604fffb0a8a9078bb24328 P 640879e300bf411c96666407655e9985: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:52:37.619731 3196 raft_consensus.cc:348] T 6625b40503604fffb0a8a9078bb24328 P b03c7542574e48749b01c123a1a93661 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "640879e300bf411c96666407655e9985" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39199 } } peers { permanent_uuid: "b03c7542574e48749b01c123a1a93661" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34205 } } peers { permanent_uuid: "6b59f952e57f4451904f3534100a4435" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 43659 } }
I20260929 10:52:37.619902 3196 raft_consensus.cc:374] T 6625b40503604fffb0a8a9078bb24328 P b03c7542574e48749b01c123a1a93661 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:52:37.619938 3196 raft_consensus.cc:729] T 6625b40503604fffb0a8a9078bb24328 P b03c7542574e48749b01c123a1a93661 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b03c7542574e48749b01c123a1a93661, State: Initialized, Role: FOLLOWER
I20260929 10:52:37.620061 3196 consensus_queue.cc:260] T 6625b40503604fffb0a8a9078bb24328 P b03c7542574e48749b01c123a1a93661 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "640879e300bf411c96666407655e9985" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39199 } } peers { permanent_uuid: "b03c7542574e48749b01c123a1a93661" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34205 } } peers { permanent_uuid: "6b59f952e57f4451904f3534100a4435" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 43659 } }
I20260929 10:52:37.620335 3196 ts_tablet_manager.cc:1428] T 6625b40503604fffb0a8a9078bb24328 P b03c7542574e48749b01c123a1a93661: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 10:52:37.620469 3053 heartbeater.cc:503] Master 127.20.80.254:37853 was elected leader, sending a full tablet report...
I20260929 10:52:37.620672 3197 raft_consensus.cc:348] T 6625b40503604fffb0a8a9078bb24328 P 640879e300bf411c96666407655e9985 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "640879e300bf411c96666407655e9985" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39199 } } peers { permanent_uuid: "b03c7542574e48749b01c123a1a93661" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34205 } } peers { permanent_uuid: "6b59f952e57f4451904f3534100a4435" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 43659 } }
I20260929 10:52:37.620846 3197 raft_consensus.cc:374] T 6625b40503604fffb0a8a9078bb24328 P 640879e300bf411c96666407655e9985 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:52:37.620893 3197 raft_consensus.cc:729] T 6625b40503604fffb0a8a9078bb24328 P 640879e300bf411c96666407655e9985 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 640879e300bf411c96666407655e9985, State: Initialized, Role: FOLLOWER
I20260929 10:52:37.621073 3197 consensus_queue.cc:260] T 6625b40503604fffb0a8a9078bb24328 P 640879e300bf411c96666407655e9985 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "640879e300bf411c96666407655e9985" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39199 } } peers { permanent_uuid: "b03c7542574e48749b01c123a1a93661" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34205 } } peers { permanent_uuid: "6b59f952e57f4451904f3534100a4435" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 43659 } }
I20260929 10:52:37.621403 3197 ts_tablet_manager.cc:1428] T 6625b40503604fffb0a8a9078bb24328 P 640879e300bf411c96666407655e9985: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 10:52:37.621416 3186 heartbeater.cc:503] Master 127.20.80.254:37853 was elected leader, sending a full tablet report...
I20260929 10:52:37.637635 2920 raft_consensus.cc:482] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:52:37.637784 2920 raft_consensus.cc:504] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "640879e300bf411c96666407655e9985" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39199 } } peers { permanent_uuid: "b03c7542574e48749b01c123a1a93661" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34205 } } peers { permanent_uuid: "6b59f952e57f4451904f3534100a4435" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 43659 } }
I20260929 10:52:37.638103 2920 leader_election.cc:288] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 640879e300bf411c96666407655e9985 (127.20.80.195:39199), b03c7542574e48749b01c123a1a93661 (127.20.80.194:34205)
I20260929 10:52:37.641553 3194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:37.638424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:37667 (local address 127.20.80.195:39199)
0929 10:52:37.638472 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:37.638475 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:37.638642 (+ 167us) server_negotiation.cc:407] Connection header received
0929 10:52:37.638669 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:37.638670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:37.638680 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:37.638751 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:52:37.638951 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.639410 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.640361 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.641130 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.641241 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:37.641250 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:37.641456 (+ 206us) server_negotiation.cc:299] Negotiation successful
0929 10:52:37.641497 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260929 10:52:37.641548 2916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:37.638265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:34205 (local address 127.20.80.193:36093)
0929 10:52:37.638352 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:37.638364 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:37.638419 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:37.638664 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:37.638665 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:37.638667 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:52:37.638781 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:37.638783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.639486 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.639487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:37.640389 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:37.640391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.641210 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.641211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:37.641299 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:37.641323 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 10:52:37.641393 (+ 70us) client_negotiation.cc:240] Negotiation successful
0929 10:52:37.641457 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":2}
I20260929 10:52:37.641548 3193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:37.638339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:36093 (local address 127.20.80.194:34205)
0929 10:52:37.638406 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:37.638409 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:37.638420 (+ 11us) server_negotiation.cc:407] Connection header received
0929 10:52:37.638515 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:37.638516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:37.638527 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:37.638592 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:52:37.638860 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.639405 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.640481 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.641124 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.641212 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:37.641222 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:37.641418 (+ 196us) server_negotiation.cc:299] Negotiation successful
0929 10:52:37.641472 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 10:52:37.641949 3202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:37.638332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:39199 (local address 127.20.80.193:37667)
0929 10:52:37.638504 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:37.638515 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:37.638581 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:37.638771 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:37.638772 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:37.638773 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:52:37.638877 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:37.638879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.639511 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.639512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:37.640282 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:37.640284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.641293 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.641294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:37.641357 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:37.641361 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:52:37.641440 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 10:52:37.641475 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":1,"thread_start_us":106,"threads_started":1}
I20260929 10:52:37.641983 3007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6625b40503604fffb0a8a9078bb24328" candidate_uuid: "6b59f952e57f4451904f3534100a4435" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b03c7542574e48749b01c123a1a93661" is_pre_election: true
I20260929 10:52:37.642156 3007 raft_consensus.cc:2454] T 6625b40503604fffb0a8a9078bb24328 P b03c7542574e48749b01c123a1a93661 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b59f952e57f4451904f3534100a4435 in term 0.
I20260929 10:52:37.642302 3140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6625b40503604fffb0a8a9078bb24328" candidate_uuid: "6b59f952e57f4451904f3534100a4435" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "640879e300bf411c96666407655e9985" is_pre_election: true
I20260929 10:52:37.642372 2800 leader_election.cc:302] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b59f952e57f4451904f3534100a4435, b03c7542574e48749b01c123a1a93661; no voters:
I20260929 10:52:37.642432 3140 raft_consensus.cc:2454] T 6625b40503604fffb0a8a9078bb24328 P 640879e300bf411c96666407655e9985 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b59f952e57f4451904f3534100a4435 in term 0.
I20260929 10:52:37.642544 2920 raft_consensus.cc:2781] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:52:37.642614 2920 raft_consensus.cc:482] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:52:37.642652 2920 raft_consensus.cc:3037] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:52:37.643404 2920 raft_consensus.cc:504] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "640879e300bf411c96666407655e9985" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39199 } } peers { permanent_uuid: "b03c7542574e48749b01c123a1a93661" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34205 } } peers { permanent_uuid: "6b59f952e57f4451904f3534100a4435" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 43659 } }
I20260929 10:52:37.643554 2920 leader_election.cc:288] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [CANDIDATE]: Term 1 election: Requested vote from peers 640879e300bf411c96666407655e9985 (127.20.80.195:39199), b03c7542574e48749b01c123a1a93661 (127.20.80.194:34205)
I20260929 10:52:37.643718 3007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6625b40503604fffb0a8a9078bb24328" candidate_uuid: "6b59f952e57f4451904f3534100a4435" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b03c7542574e48749b01c123a1a93661"
I20260929 10:52:37.643735 3140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6625b40503604fffb0a8a9078bb24328" candidate_uuid: "6b59f952e57f4451904f3534100a4435" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "640879e300bf411c96666407655e9985"
I20260929 10:52:37.643807 3007 raft_consensus.cc:3037] T 6625b40503604fffb0a8a9078bb24328 P b03c7542574e48749b01c123a1a93661 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:52:37.643808 3140 raft_consensus.cc:3037] T 6625b40503604fffb0a8a9078bb24328 P 640879e300bf411c96666407655e9985 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:52:37.644680 3007 raft_consensus.cc:2454] T 6625b40503604fffb0a8a9078bb24328 P b03c7542574e48749b01c123a1a93661 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b59f952e57f4451904f3534100a4435 in term 1.
I20260929 10:52:37.644680 3140 raft_consensus.cc:2454] T 6625b40503604fffb0a8a9078bb24328 P 640879e300bf411c96666407655e9985 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b59f952e57f4451904f3534100a4435 in term 1.
I20260929 10:52:37.644840 2800 leader_election.cc:302] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b59f952e57f4451904f3534100a4435, b03c7542574e48749b01c123a1a93661; no voters:
I20260929 10:52:37.644936 2920 raft_consensus.cc:2781] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:52:37.645113 2920 raft_consensus.cc:686] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [term 1 LEADER]: Becoming Leader. State: Replica: 6b59f952e57f4451904f3534100a4435, State: Running, Role: LEADER
I20260929 10:52:37.645174 2920 consensus_queue.cc:237] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "640879e300bf411c96666407655e9985" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39199 } } peers { permanent_uuid: "b03c7542574e48749b01c123a1a93661" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34205 } } peers { permanent_uuid: "6b59f952e57f4451904f3534100a4435" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 43659 } }
I20260929 10:52:37.646027 2725 catalog_manager.cc:5495] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b59f952e57f4451904f3534100a4435 (127.20.80.193). New cstate: current_term: 1 leader_uuid: "6b59f952e57f4451904f3534100a4435" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "640879e300bf411c96666407655e9985" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b03c7542574e48749b01c123a1a93661" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b59f952e57f4451904f3534100a4435" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 43659 } health_report { overall_health: HEALTHY } } }
I20260929 10:52:37.657298 2922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:37.654824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46952 (local address 127.20.80.193:43659)
0929 10:52:37.654864 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:37.654867 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:37.654875 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:52:37.654955 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:37.654956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:37.654977 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:37.655054 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:52:37.655198 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.655684 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.656426 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:37.656640 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:37.656727 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:37.656905 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:52:37.657063 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:52:37.657151 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:37.657189 (+ 38us) server_negotiation.cc:299] Negotiation successful
0929 10:52:37.657215 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 10:52:37.658807 3140 raft_consensus.cc:1261] T 6625b40503604fffb0a8a9078bb24328 P 640879e300bf411c96666407655e9985 [term 1 FOLLOWER]: Refusing update from remote peer 6b59f952e57f4451904f3534100a4435: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:52:37.658820 3007 raft_consensus.cc:1261] T 6625b40503604fffb0a8a9078bb24328 P b03c7542574e48749b01c123a1a93661 [term 1 FOLLOWER]: Refusing update from remote peer 6b59f952e57f4451904f3534100a4435: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:52:37.659132 2920 consensus_queue.cc:1035] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [LEADER]: Connected to new peer: Peer: permanent_uuid: "640879e300bf411c96666407655e9985" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:52:37.659297 3203 consensus_queue.cc:1035] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b03c7542574e48749b01c123a1a93661" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:52:37.662230 3206 mvcc.cc:205] Tried to move back new op lower bound from 7334621829767913472 to 7334621829715169280. Current Snapshot: MvccSnapshot[applied={T|T < 7334621829767913472}]
I20260929 10:52:37.662417 3207 mvcc.cc:205] Tried to move back new op lower bound from 7334621829767913472 to 7334621829715169280. Current Snapshot: MvccSnapshot[applied={T|T < 7334621829767913472}]
I20260929 10:52:37.663515 3209 mvcc.cc:205] Tried to move back new op lower bound from 7334621829767913472 to 7334621829715169280. Current Snapshot: MvccSnapshot[applied={T|T < 7334621829767913472 or (T in {7334621829767913472})}]
I20260929 10:52:37.664225 2725 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:35108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:52:37.664322 2725 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:35108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:52:37.665038 2725 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96: Sending DeleteTablet for 3 replicas of tablet 6625b40503604fffb0a8a9078bb24328
I20260929 10:52:37.665374 2848 tablet_service.cc:1514] Processing DeleteTablet for tablet 6625b40503604fffb0a8a9078bb24328 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:52:37 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:46936
I20260929 10:52:37.665354 2987 tablet_service.cc:1514] Processing DeleteTablet for tablet 6625b40503604fffb0a8a9078bb24328 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:52:37 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:43962
I20260929 10:52:37.665463 2923 tablet_replica.cc:330] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435: stopping tablet replica
I20260929 10:52:37.665510 2923 raft_consensus.cc:2229] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:52:37.665591 2923 raft_consensus.cc:2258] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:52:37.665577 3120 tablet_service.cc:1514] Processing DeleteTablet for tablet 6625b40503604fffb0a8a9078bb24328 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:52:37 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:45170
I20260929 10:52:37.665853 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 2763
I20260929 10:52:37.665894 2923 ts_tablet_manager.cc:1905] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:52:37.666734 3216 tablet_replica.cc:330] T 6625b40503604fffb0a8a9078bb24328 P b03c7542574e48749b01c123a1a93661: stopping tablet replica
I20260929 10:52:37.666842 3216 raft_consensus.cc:2229] T 6625b40503604fffb0a8a9078bb24328 P b03c7542574e48749b01c123a1a93661 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:52:37.666960 3216 raft_consensus.cc:2258] T 6625b40503604fffb0a8a9078bb24328 P b03c7542574e48749b01c123a1a93661 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:52:37.667344 3216 ts_tablet_manager.cc:1905] T 6625b40503604fffb0a8a9078bb24328 P b03c7542574e48749b01c123a1a93661: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:52:37.668458 3218 tablet_replica.cc:330] T 6625b40503604fffb0a8a9078bb24328 P 640879e300bf411c96666407655e9985: stopping tablet replica
I20260929 10:52:37.668538 3218 raft_consensus.cc:2229] T 6625b40503604fffb0a8a9078bb24328 P 640879e300bf411c96666407655e9985 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:52:37.668658 3218 raft_consensus.cc:2258] T 6625b40503604fffb0a8a9078bb24328 P 640879e300bf411c96666407655e9985 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:52:37.669085 3216 ts_tablet_manager.cc:1918] T 6625b40503604fffb0a8a9078bb24328 P b03c7542574e48749b01c123a1a93661: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:52:37.669130 3218 ts_tablet_manager.cc:1905] T 6625b40503604fffb0a8a9078bb24328 P 640879e300bf411c96666407655e9985: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:52:37.669162 3216 log.cc:1196] T 6625b40503604fffb0a8a9078bb24328 P b03c7542574e48749b01c123a1a93661: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/wal/wals/6625b40503604fffb0a8a9078bb24328
I20260929 10:52:37.669463 3216 ts_tablet_manager.cc:1939] T 6625b40503604fffb0a8a9078bb24328 P b03c7542574e48749b01c123a1a93661: Deleting consensus metadata
I20260929 10:52:37.669884 2708 catalog_manager.cc:4841] TS b03c7542574e48749b01c123a1a93661 (127.20.80.194:34205): tablet 6625b40503604fffb0a8a9078bb24328 (table test-table [id=2235ef1a993342e3ae149af5662f0d9c]) successfully deleted
I20260929 10:52:37.670679 3218 ts_tablet_manager.cc:1918] T 6625b40503604fffb0a8a9078bb24328 P 640879e300bf411c96666407655e9985: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:52:37.670764 3218 log.cc:1196] T 6625b40503604fffb0a8a9078bb24328 P 640879e300bf411c96666407655e9985: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/wal/wals/6625b40503604fffb0a8a9078bb24328
I20260929 10:52:37.671046 3218 ts_tablet_manager.cc:1939] T 6625b40503604fffb0a8a9078bb24328 P 640879e300bf411c96666407655e9985: Deleting consensus metadata
I20260929 10:52:37.671424 2709 catalog_manager.cc:4841] TS 640879e300bf411c96666407655e9985 (127.20.80.195:39199): tablet 6625b40503604fffb0a8a9078bb24328 (table test-table [id=2235ef1a993342e3ae149af5662f0d9c]) successfully deleted
W20260929 10:52:37.672263 2708 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.80.193:43659 (error 108)
I20260929 10:52:37.672603 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 2919
I20260929 10:52:37.672772 2921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:37.672625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:43659 (local address 127.0.0.1:46958)
0929 10:52:37.672698 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:37.672736 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.193:43659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260929 10:52:37.677122 2708 catalog_manager.cc:4584] TS 6b59f952e57f4451904f3534100a4435 (127.20.80.193:43659): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6625b40503604fffb0a8a9078bb24328: Network error: Client connection negotiation failed: client connection to 127.20.80.193:43659: connect: Connection refused (error 111)
I20260929 10:52:37.677821 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 3056
I20260929 10:52:37.682739 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 2691
I20260929 10:52:37.688370 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:37853
--webserver_interface=127.20.80.254
--webserver_port=33955
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:33539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:37853
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:52:37.768635 3220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:52:37.768831 3220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:52:37.768853 3220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:52:37.770360 3220 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:52:37.770413 3220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:52:37.770429 3220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:52:37.770442 3220 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:52:37.770455 3220 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:52:37.772079 3220 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:33539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:37853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:37853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=33955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:52:37.772269 3220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:52:37.772450 3220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:52:37.777316 3225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:37.777406 3228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:37.777316 3226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:37.777477 3220 server_base.cc:1034] running on GCE node
I20260929 10:52:37.777700 3220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:52:37.777966 3220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:52:37.779137 3220 hybrid_clock.cc:648] HybridClock initialized: now 1790679157779141 us; error 44 us; skew 500 ppm
Sep 29 10:52:37 dist-test-slave-vqch krb5kdc[2140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679157, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:52:37.781829 3220 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:52:37.782356 3220 webserver.cc:460] Webserver started at http://127.20.80.254:33955/ using document root <none> and password file <none>
I20260929 10:52:37.782598 3220 fs_manager.cc:359] Metadata directory not provided
I20260929 10:52:37.782666 3220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:52:37.783972 3220 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:52:37.784583 3235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:37.784760 3220 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:52:37.784860 3220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "1ab3cc1989514145aad5dfa61e847b96"
format_stamp: "Formatted at 2026-09-29 10:52:36 on dist-test-slave-vqch"
server_key: "61726378975ad41bd47ed6428652b572"
server_key_iv: "a271787484b42854562d28a652b506e3"
server_key_version: "encryptionkey@0"
I20260929 10:52:37.785267 3220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:52:37.801447 3220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:52:37.803905 3220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:52:37.804072 3220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:52:37.808215 3220 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:37853
I20260929 10:52:37.808244 3287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:37853 every 8 connection(s)
I20260929 10:52:37.808661 3220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:52:37.810506 3288 sys_catalog.cc:263] Verifying existing consensus state
I20260929 10:52:37.811141 3288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96: Bootstrap starting.
I20260929 10:52:37.813256 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 3220
I20260929 10:52:37.813534 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.193:43659
--local_ip_for_outbound_sockets=127.20.80.193
--tserver_master_addrs=127.20.80.254:37853
--webserver_port=46475
--webserver_interface=127.20.80.193
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:33539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:52:37.820034 3288 log.cc:824] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96: Log is configured to *not* fsync() on all Append() calls
I20260929 10:52:37.822414 3288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 10:52:37.822737 3288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96: Bootstrap complete.
I20260929 10:52:37.825079 3288 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab3cc1989514145aad5dfa61e847b96" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37853 } }
I20260929 10:52:37.825354 3288 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ab3cc1989514145aad5dfa61e847b96, State: Initialized, Role: FOLLOWER
I20260929 10:52:37.825544 3288 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab3cc1989514145aad5dfa61e847b96" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37853 } }
I20260929 10:52:37.825624 3288 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:52:37.825677 3288 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:52:37.825734 3288 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [term 2 FOLLOWER]: Advancing to term 3
I20260929 10:52:37.826717 3288 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab3cc1989514145aad5dfa61e847b96" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37853 } }
I20260929 10:52:37.826839 3288 leader_election.cc:302] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ab3cc1989514145aad5dfa61e847b96; no voters:
I20260929 10:52:37.827045 3288 leader_election.cc:288] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260929 10:52:37.827081 3292 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [term 3 FOLLOWER]: Leader election won for term 3
I20260929 10:52:37.827188 3292 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [term 3 LEADER]: Becoming Leader. State: Replica: 1ab3cc1989514145aad5dfa61e847b96, State: Running, Role: LEADER
I20260929 10:52:37.827282 3292 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab3cc1989514145aad5dfa61e847b96" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37853 } }
I20260929 10:52:37.827318 3288 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:52:37.827550 3292 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ab3cc1989514145aad5dfa61e847b96. Latest consensus state: current_term: 3 leader_uuid: "1ab3cc1989514145aad5dfa61e847b96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab3cc1989514145aad5dfa61e847b96" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37853 } } }
I20260929 10:52:37.827628 3292 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [sys.catalog]: This master's current role is: LEADER
I20260929 10:52:37.827556 3293 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "1ab3cc1989514145aad5dfa61e847b96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab3cc1989514145aad5dfa61e847b96" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37853 } } }
I20260929 10:52:37.827826 3296 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:52:37.827834 3293 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ab3cc1989514145aad5dfa61e847b96 [sys.catalog]: This master's current role is: LEADER
I20260929 10:52:37.828684 3296 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:52:37.828805 3296 catalog_manager.cc:1259] Loaded cluster ID: 52e1580f6b814e2d8b2a82cec0f7b7e5
I20260929 10:52:37.828848 3296 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260929 10:52:37.829134 3296 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f6beaf659ad google::LogMessage::Fail() at ??:0
@ 0x7f6beaf6888a google::LogMessage::SendToLog() at ??:0
@ 0x7f6beaf654de google::LogMessage::Flush() at ??:0
@ 0x7f6beaf66a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f6bf2e860f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f6bf2e8f8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6bece330d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6bece28954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6bedc8c6db start_thread at ??:0
@ 0x7f6be9a6361f clone at ??:0
W20260929 10:52:37.919472 3290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:52:37.919664 3290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:52:37.919690 3290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:52:37.921129 3290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:52:37.921185 3290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:52:37.921224 3290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.193
I20260929 10:52:37.923113 3290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:33539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.193:43659
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.80.193
--webserver_port=46475
--tserver_master_addrs=127.20.80.254:37853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.193
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:52:37.923368 3290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:52:37.923630 3290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:52:37.929149 3324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:37.929339 3327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:37.929173 3325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:37.929499 3290 server_base.cc:1034] running on GCE node
I20260929 10:52:37.929708 3290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:52:37.929981 3290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:52:37.931151 3290 hybrid_clock.cc:648] HybridClock initialized: now 1790679157931125 us; error 41 us; skew 500 ppm
Sep 29 10:52:37 dist-test-slave-vqch krb5kdc[2140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679157, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:52:37.934366 3290 init.cc:377] Logged in from keytab as kudu/127.20.80.193@KRBTEST.COM (short username kudu)
I20260929 10:52:37.934861 3290 webserver.cc:460] Webserver started at http://127.20.80.193:46475/ using document root <none> and password file <none>
I20260929 10:52:37.935106 3290 fs_manager.cc:359] Metadata directory not provided
I20260929 10:52:37.935168 3290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:52:37.936946 3290 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 10:52:37.938021 3334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:37.938196 3290 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:52:37.938293 3290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/wal
uuid: "6b59f952e57f4451904f3534100a4435"
format_stamp: "Formatted at 2026-09-29 10:52:36 on dist-test-slave-vqch"
server_key: "6bcb6adfd5e3334122d7e4e26f49e70d"
server_key_iv: "ca704924dffe88c3936aaf7d35064945"
server_key_version: "encryptionkey@0"
I20260929 10:52:37.938742 3290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:52:37.951437 3290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:52:37.953634 3290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:52:37.953783 3290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:52:37.954034 3290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:52:37.954511 3341 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 10:52:37.956182 3290 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 10:52:37.956252 3290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260929 10:52:37.956301 3290 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 10:52:37.956931 3290 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 10:52:37.956991 3290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260929 10:52:37.957085 3341 tablet_bootstrap.cc:492] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435: Bootstrap starting.
I20260929 10:52:37.958995 3341 log.cc:824] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435: Log is configured to *not* fsync() on all Append() calls
I20260929 10:52:37.962086 3341 tablet_bootstrap.cc:492] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 10:52:37.962387 3341 tablet_bootstrap.cc:492] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435: Bootstrap complete.
I20260929 10:52:37.962546 3341 ts_tablet_manager.cc:1397] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 10:52:37.964257 3290 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.193:43659
I20260929 10:52:37.964321 3449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.193:43659 every 8 connection(s)
I20260929 10:52:37.964618 3290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
I20260929 10:52:37.964375 3341 raft_consensus.cc:348] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "640879e300bf411c96666407655e9985" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39199 } } peers { permanent_uuid: "b03c7542574e48749b01c123a1a93661" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34205 } } peers { permanent_uuid: "6b59f952e57f4451904f3534100a4435" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 43659 } }
I20260929 10:52:37.964720 3341 raft_consensus.cc:729] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b59f952e57f4451904f3534100a4435, State: Initialized, Role: FOLLOWER
I20260929 10:52:37.964881 3341 consensus_queue.cc:260] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "640879e300bf411c96666407655e9985" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39199 } } peers { permanent_uuid: "b03c7542574e48749b01c123a1a93661" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34205 } } peers { permanent_uuid: "6b59f952e57f4451904f3534100a4435" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 43659 } }
I20260929 10:52:37.965293 3341 ts_tablet_manager.cc:1428] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260929 10:52:37.969106 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 3290
I20260929 10:52:37.969388 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.194:34205
--local_ip_for_outbound_sockets=127.20.80.194
--tserver_master_addrs=127.20.80.254:37853
--webserver_port=35347
--webserver_interface=127.20.80.194
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:33539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:52:38.072451 3454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:52:38.072642 3454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:52:38.072665 3454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:52:38.073881 3454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:52:38.073922 3454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:52:38.073948 3454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.194
I20260929 10:52:38.075357 3454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:33539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.194:34205
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.80.194
--webserver_port=35347
--tserver_master_addrs=127.20.80.254:37853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.194
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:52:38.075593 3454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:52:38.075841 3454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:52:38.080889 3461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:38.080919 3463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:38.081241 3454 server_base.cc:1034] running on GCE node
W20260929 10:52:38.080919 3460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:38.081496 3454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:52:38.081743 3454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:52:38.082907 3454 hybrid_clock.cc:648] HybridClock initialized: now 1790679158082885 us; error 41 us; skew 500 ppm
Sep 29 10:52:38 dist-test-slave-vqch krb5kdc[2140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679158, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:52:38.085274 3454 init.cc:377] Logged in from keytab as kudu/127.20.80.194@KRBTEST.COM (short username kudu)
I20260929 10:52:38.085729 3454 webserver.cc:460] Webserver started at http://127.20.80.194:35347/ using document root <none> and password file <none>
I20260929 10:52:38.085987 3454 fs_manager.cc:359] Metadata directory not provided
I20260929 10:52:38.086058 3454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:52:38.087424 3454 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:52:38.088104 3470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:38.088284 3454 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 10:52:38.088392 3454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/wal
uuid: "b03c7542574e48749b01c123a1a93661"
format_stamp: "Formatted at 2026-09-29 10:52:36 on dist-test-slave-vqch"
server_key: "65b0caa4c3eea46eeb1479b88b780f00"
server_key_iv: "2683f4a25f1de177be466753e13f1ce5"
server_key_version: "encryptionkey@0"
I20260929 10:52:38.088773 3454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:52:38.106053 3454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:52:38.108250 3454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:52:38.108400 3454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:52:38.108654 3454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:52:38.108966 3454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:52:38.109081 3454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:38.109105 3454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:52:38.109148 3454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:38.115528 3454 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.194:34205
I20260929 10:52:38.115574 3585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.194:34205 every 8 connection(s)
I20260929 10:52:38.115921 3454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
I20260929 10:52:38.125408 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 3454
I20260929 10:52:38.125689 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.195:39199
--local_ip_for_outbound_sockets=127.20.80.195
--tserver_master_addrs=127.20.80.254:37853
--webserver_port=34129
--webserver_interface=127.20.80.195
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:33539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:52:38.177733 3453 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:52:37.965290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.193:46553)
0929 10:52:37.965457 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:37.965506 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:37.965624 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:38.177544 (+211920us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":72,"threads_started":1}
W20260929 10:52:38.178001 3450 heartbeater.cc:644] Failed to heartbeat to 127.20.80.254:37853 (0 consecutive failures): Network error: Failed to ping master at 127.20.80.254:37853: Client connection negotiation failed: client connection to 127.20.80.254:37853: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260929 10:52:38.178215 3588 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:52:38.116567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.194:37475)
0929 10:52:38.116742 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:38.116777 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:38.117293 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:38.177546 (+ 60253us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":66,"threads_started":1}
I20260929 10:52:38.178365 3453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:38.178203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.193:46667)
0929 10:52:38.178276 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:38.178333 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260929 10:52:38.178546 3586 heartbeater.cc:644] Failed to heartbeat to 127.20.80.254:37853 (0 consecutive failures): Network error: Failed to ping master at 127.20.80.254:37853: Client connection negotiation failed: client connection to 127.20.80.254:37853: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 10:52:38.178712 3453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:38.178625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.193:42271)
0929 10:52:38.178665 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:38.178687 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260929 10:52:38.178822 3450 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 10:52:38.178921 3588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:38.178788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.194:56169)
0929 10:52:38.178853 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:38.178894 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260929 10:52:38.179282 3588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:38.179195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.194:59385)
0929 10:52:38.179243 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:38.179266 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260929 10:52:38.179411 3586 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 10:52:38.222783 3589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:52:38.223017 3589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:52:38.223055 3589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:52:38.224197 3589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:52:38.224268 3589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:52:38.224313 3589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.195
I20260929 10:52:38.225786 3589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:33539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.195:39199
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.80.195
--webserver_port=34129
--tserver_master_addrs=127.20.80.254:37853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.195
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:52:38.226046 3589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:52:38.226284 3589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:52:38.231266 3595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:38.231273 3597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:38.231446 3589 server_base.cc:1034] running on GCE node
W20260929 10:52:38.231310 3594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:38.231673 3589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:52:38.231870 3589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:52:38.233044 3589 hybrid_clock.cc:648] HybridClock initialized: now 1790679158233033 us; error 27 us; skew 500 ppm
Sep 29 10:52:38 dist-test-slave-vqch krb5kdc[2140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679158, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:52:38.235538 3589 init.cc:377] Logged in from keytab as kudu/127.20.80.195@KRBTEST.COM (short username kudu)
I20260929 10:52:38.235947 3589 webserver.cc:460] Webserver started at http://127.20.80.195:34129/ using document root <none> and password file <none>
I20260929 10:52:38.236168 3589 fs_manager.cc:359] Metadata directory not provided
I20260929 10:52:38.236248 3589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:52:38.237665 3589 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:52:38.238286 3604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:38.238466 3589 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:52:38.238567 3589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/wal
uuid: "640879e300bf411c96666407655e9985"
format_stamp: "Formatted at 2026-09-29 10:52:36 on dist-test-slave-vqch"
server_key: "cc356bc2c12fcda8cb3e86049eba111e"
server_key_iv: "313662a96148ee583226440d8020aacd"
server_key_version: "encryptionkey@0"
I20260929 10:52:38.238903 3589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:52:38.247014 3589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:52:38.249330 3589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:52:38.249486 3589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:52:38.249702 3589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:52:38.249995 3589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:52:38.250053 3589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:38.250092 3589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:52:38.250119 3589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:38.255712 3589 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.195:39199
I20260929 10:52:38.255808 3718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.195:39199 every 8 connection(s)
I20260929 10:52:38.256122 3589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
I20260929 10:52:38.257210 3721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:38.256805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.195:56001)
0929 10:52:38.256953 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:38.257076 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":70,"threads_started":1}
W20260929 10:52:38.257620 3719 heartbeater.cc:644] Failed to heartbeat to 127.20.80.254:37853 (0 consecutive failures): Network error: Failed to ping master at 127.20.80.254:37853: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
I20260929 10:52:38.257890 3721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:38.257799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.195:39977)
0929 10:52:38.257846 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:38.257872 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260929 10:52:38.258159 3721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:38.258082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.195:50163)
0929 10:52:38.258123 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:38.258145 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260929 10:52:38.258277 3719 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 10:52:38.261444 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 3589
I20260929 10:52:39.179382 3723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:39.179119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.193:57545)
0929 10:52:39.179279 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:39.179317 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":64,"threads_started":1}
I20260929 10:52:39.179929 3724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:39.179718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.194:36263)
0929 10:52:39.179824 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:39.179860 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":66,"threads_started":1}
I20260929 10:52:39.220311 3725 raft_consensus.cc:482] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:52:39.220429 3725 raft_consensus.cc:504] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "640879e300bf411c96666407655e9985" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39199 } } peers { permanent_uuid: "b03c7542574e48749b01c123a1a93661" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34205 } } peers { permanent_uuid: "6b59f952e57f4451904f3534100a4435" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 43659 } }
I20260929 10:52:39.220956 3725 leader_election.cc:288] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 640879e300bf411c96666407655e9985 (127.20.80.195:39199), b03c7542574e48749b01c123a1a93661 (127.20.80.194:34205)
Sep 29 10:52:39 dist-test-slave-vqch krb5kdc[2140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679157, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for kudu/127.20.80.195@KRBTEST.COM
Sep 29 10:52:39 dist-test-slave-vqch krb5kdc[2140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679157, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for kudu/127.20.80.194@KRBTEST.COM
I20260929 10:52:39.231216 3723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:39.221111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:39199 (local address 127.20.80.193:48907)
0929 10:52:39.221163 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:39.221174 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:39.221229 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:39.222081 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:39.222082 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:39.222640 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:52:39.222709 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:39.222712 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:39.223850 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:39.223851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:39.224381 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:39.224383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:39.225007 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:39.225008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:39.225104 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:39.225720 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:52:39.225731 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:52:39.227253 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:52:39.229507 (+ 2254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:39.229515 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:39.229517 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:39.229819 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:39.230093 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:39.230094 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:39.230094 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:39.230224 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:39.230664 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:52:39.230671 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:52:39.230814 (+ 143us) client_negotiation.cc:769] Sending connection context
0929 10:52:39.230922 (+ 108us) client_negotiation.cc:240] Negotiation successful
0929 10:52:39.231095 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260929 10:52:39.231216 3728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:39.221356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:54719 (local address 127.20.80.194:34205)
0929 10:52:39.221496 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:39.221500 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:39.221510 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:52:39.221601 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:39.221602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:39.221621 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:39.221730 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:39.222464 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:39.223439 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:39.224435 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:39.224843 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:39.224968 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:39.227699 (+ 2731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:39.227703 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:39.227703 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:39.227714 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:39.229688 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:39.230100 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:39.230103 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:39.230103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:39.230203 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:39.230438 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:39.230439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:39.230439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:39.230710 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:39.230835 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:39.230997 (+ 162us) server_negotiation.cc:299] Negotiation successful
0929 10:52:39.231119 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":84,"threads_started":1}
I20260929 10:52:39.231472 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:39.221251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:48907 (local address 127.20.80.195:39199)
0929 10:52:39.221411 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:39.221428 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:39.221448 (+ 20us) server_negotiation.cc:407] Connection header received
0929 10:52:39.221888 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:39.221890 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:39.221944 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:39.222106 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:39.222833 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:39.223739 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:39.224476 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:39.224885 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:39.225027 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:39.227377 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:39.227381 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:39.227382 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:39.227395 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:39.229418 (+ 2023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:39.229900 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:39.229904 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:39.229904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:39.230015 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:39.230314 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:39.230314 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:39.230315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:39.230589 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:39.230648 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:39.230999 (+ 351us) server_negotiation.cc:299] Negotiation successful
0929 10:52:39.231185 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":77,"threads_started":1}
I20260929 10:52:39.231480 3726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:39.221150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:34205 (local address 127.20.80.193:54719)
0929 10:52:39.221256 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:39.221265 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:39.221306 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:39.221792 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:39.221795 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:39.222112 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:52:39.222326 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:39.222332 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:39.223555 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:39.223556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:39.224326 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:39.224328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:39.224940 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:39.224941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:39.225100 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:39.225774 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:52:39.225779 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:52:39.227592 (+ 1813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:52:39.229784 (+ 2192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:39.229784 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:39.229785 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:39.230005 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:39.230297 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:39.230297 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:39.230298 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:39.230367 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:39.230816 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:52:39.230817 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:52:39.230854 (+ 37us) client_negotiation.cc:769] Sending connection context
0929 10:52:39.230942 (+ 88us) client_negotiation.cc:240] Negotiation successful
0929 10:52:39.231281 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":197,"thread_start_us":44,"threads_started":1}
W20260929 10:52:39.232236 3336 leader_election.cc:341] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 640879e300bf411c96666407655e9985 (127.20.80.195:39199): Not found: Tablet not found: 6625b40503604fffb0a8a9078bb24328
W20260929 10:52:39.232640 3335 leader_election.cc:341] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b03c7542574e48749b01c123a1a93661 (127.20.80.194:34205): Not found: Tablet not found: 6625b40503604fffb0a8a9078bb24328
I20260929 10:52:39.232687 3335 leader_election.cc:302] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b59f952e57f4451904f3534100a4435; no voters: 640879e300bf411c96666407655e9985, b03c7542574e48749b01c123a1a93661
I20260929 10:52:39.232851 3725 raft_consensus.cc:2726] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 10:52:39.258833 3729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:39.258596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.195:57079)
0929 10:52:39.258745 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:39.258795 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":88,"threads_started":1}
I20260929 10:52:40.180182 3730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:40.179936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.193:44507)
0929 10:52:40.180094 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:40.180135 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":81,"threads_started":1}
I20260929 10:52:40.180624 3731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:40.180430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.194:41775)
0929 10:52:40.180530 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:40.180559 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":62,"threads_started":1}
I20260929 10:52:40.259478 3732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:40.259289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.195:48193)
0929 10:52:40.259407 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:40.259441 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":62,"threads_started":1}
I20260929 10:52:41.181322 3734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:41.181101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.194:58859)
0929 10:52:41.181250 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:41.181283 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":63,"threads_started":1}
I20260929 10:52:41.181308 3733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:41.181029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.193:42041)
0929 10:52:41.181192 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:41.181238 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":103,"threads_started":1}
I20260929 10:52:41.225248 3735 raft_consensus.cc:482] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:52:41.225343 3735 raft_consensus.cc:504] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "640879e300bf411c96666407655e9985" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39199 } } peers { permanent_uuid: "b03c7542574e48749b01c123a1a93661" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34205 } } peers { permanent_uuid: "6b59f952e57f4451904f3534100a4435" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 43659 } }
I20260929 10:52:41.225521 3735 leader_election.cc:288] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 640879e300bf411c96666407655e9985 (127.20.80.195:39199), b03c7542574e48749b01c123a1a93661 (127.20.80.194:34205)
W20260929 10:52:41.225960 3335 leader_election.cc:341] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b03c7542574e48749b01c123a1a93661 (127.20.80.194:34205): Not found: Tablet not found: 6625b40503604fffb0a8a9078bb24328
W20260929 10:52:41.226045 3336 leader_election.cc:341] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 640879e300bf411c96666407655e9985 (127.20.80.195:39199): Not found: Tablet not found: 6625b40503604fffb0a8a9078bb24328
I20260929 10:52:41.226132 3336 leader_election.cc:302] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b59f952e57f4451904f3534100a4435; no voters: 640879e300bf411c96666407655e9985, b03c7542574e48749b01c123a1a93661
I20260929 10:52:41.226217 3735 raft_consensus.cc:2726] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 10:52:41.260274 3736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:41.260005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.195:43489)
0929 10:52:41.260183 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:41.260232 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":89,"threads_started":1}
I20260929 10:52:42.182024 3737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:42.181773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.193:42639)
0929 10:52:42.181930 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:42.181968 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":78,"threads_started":1}
I20260929 10:52:42.182049 3738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:42.181835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.194:58325)
0929 10:52:42.181977 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:42.182010 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":87,"threads_started":1}
I20260929 10:52:42.261144 3739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:42.260858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.195:40683)
0929 10:52:42.261066 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:42.261105 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":82,"threads_started":1}
I20260929 10:52:43.182721 3740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:43.182477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.193:50419)
0929 10:52:43.182619 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:43.182661 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":60,"threads_started":1}
I20260929 10:52:43.182768 3741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:43.182565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.194:42775)
0929 10:52:43.182694 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:43.182727 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":67,"threads_started":1}
I20260929 10:52:43.261890 3742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:43.261648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.195:58445)
0929 10:52:43.261805 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:43.261846 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":57,"threads_started":1}
I20260929 10:52:43.746899 3743 raft_consensus.cc:482] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:52:43.746990 3743 raft_consensus.cc:504] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "640879e300bf411c96666407655e9985" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39199 } } peers { permanent_uuid: "b03c7542574e48749b01c123a1a93661" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34205 } } peers { permanent_uuid: "6b59f952e57f4451904f3534100a4435" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 43659 } }
I20260929 10:52:43.747195 3743 leader_election.cc:288] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 640879e300bf411c96666407655e9985 (127.20.80.195:39199), b03c7542574e48749b01c123a1a93661 (127.20.80.194:34205)
W20260929 10:52:43.747563 3335 leader_election.cc:341] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b03c7542574e48749b01c123a1a93661 (127.20.80.194:34205): Not found: Tablet not found: 6625b40503604fffb0a8a9078bb24328
W20260929 10:52:43.747641 3336 leader_election.cc:341] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 640879e300bf411c96666407655e9985 (127.20.80.195:39199): Not found: Tablet not found: 6625b40503604fffb0a8a9078bb24328
I20260929 10:52:43.747685 3336 leader_election.cc:302] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b59f952e57f4451904f3534100a4435; no voters: 640879e300bf411c96666407655e9985, b03c7542574e48749b01c123a1a93661
I20260929 10:52:43.747780 3743 raft_consensus.cc:2726] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 10:52:44.183423 3744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:44.183172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.193:51627)
0929 10:52:44.183317 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:44.183357 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":67,"threads_started":1}
I20260929 10:52:44.183561 3745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:44.183346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.194:32875)
0929 10:52:44.183463 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:44.183497 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":60,"threads_started":1}
I20260929 10:52:44.262598 3746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:44.262396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.195:40239)
0929 10:52:44.262517 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:44.262554 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":58,"threads_started":1}
I20260929 10:52:45.184211 3747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:45.183989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.193:51007)
0929 10:52:45.184123 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:45.184165 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":78,"threads_started":1}
I20260929 10:52:45.184265 3748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:45.184042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.194:41265)
0929 10:52:45.184178 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:45.184222 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":76,"threads_started":1}
I20260929 10:52:45.263293 3749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:45.263079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.195:42817)
0929 10:52:45.263217 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:45.263249 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":79,"threads_started":1}
I20260929 10:52:46.184957 3750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:46.184729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.193:33503)
0929 10:52:46.184872 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:46.184910 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":59,"threads_started":1}
I20260929 10:52:46.185081 3751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:46.184736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.194:44171)
0929 10:52:46.184934 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:46.185011 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":85,"threads_started":1}
I20260929 10:52:46.264032 3752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:46.263767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.195:46125)
0929 10:52:46.263927 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:46.263966 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":65,"threads_started":1}
I20260929 10:52:47.185652 3753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:47.185424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.193:35467)
0929 10:52:47.185571 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:47.185607 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":95,"threads_started":1}
I20260929 10:52:47.185806 3754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:47.185563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.194:39071)
0929 10:52:47.185677 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:47.185739 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":70,"threads_started":1}
I20260929 10:52:47.264775 3755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:47.264533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.195:60795)
0929 10:52:47.264667 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:47.264734 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":64,"threads_started":1}
I20260929 10:52:48.186373 3756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:48.186135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.193:48083)
0929 10:52:48.186288 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:48.186325 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
I20260929 10:52:48.186417 3757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:48.186212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.194:39705)
0929 10:52:48.186339 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:48.186376 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":79,"threads_started":1}
I20260929 10:52:48.199222 3758 raft_consensus.cc:482] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:52:48.199319 3758 raft_consensus.cc:504] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "640879e300bf411c96666407655e9985" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39199 } } peers { permanent_uuid: "b03c7542574e48749b01c123a1a93661" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34205 } } peers { permanent_uuid: "6b59f952e57f4451904f3534100a4435" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 43659 } }
I20260929 10:52:48.199481 3758 leader_election.cc:288] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 640879e300bf411c96666407655e9985 (127.20.80.195:39199), b03c7542574e48749b01c123a1a93661 (127.20.80.194:34205)
W20260929 10:52:48.199904 3336 leader_election.cc:341] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 640879e300bf411c96666407655e9985 (127.20.80.195:39199): Not found: Tablet not found: 6625b40503604fffb0a8a9078bb24328
W20260929 10:52:48.200057 3335 leader_election.cc:341] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b03c7542574e48749b01c123a1a93661 (127.20.80.194:34205): Not found: Tablet not found: 6625b40503604fffb0a8a9078bb24328
I20260929 10:52:48.200120 3335 leader_election.cc:302] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b59f952e57f4451904f3534100a4435; no voters: 640879e300bf411c96666407655e9985, b03c7542574e48749b01c123a1a93661
I20260929 10:52:48.200212 3758 raft_consensus.cc:2726] T 6625b40503604fffb0a8a9078bb24328 P 6b59f952e57f4451904f3534100a4435 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 10:52:48.265442 3759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:48.265227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.195:40061)
0929 10:52:48.265349 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:48.265388 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":58,"threads_started":1}
I20260929 10:52:49.187104 3760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:49.186859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.193:58269)
0929 10:52:49.186997 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:49.187039 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":58,"threads_started":1}
I20260929 10:52:49.187177 3761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:49.186958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.194:43511)
0929 10:52:49.187099 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:49.187133 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":83,"threads_started":1}
I20260929 10:52:49.266122 3762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:49.265935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.195:57503)
0929 10:52:49.266047 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:49.266081 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":58,"threads_started":1}
I20260929 10:52:50.187866 3764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:50.187646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.194:56249)
0929 10:52:50.187783 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:50.187820 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":59,"threads_started":1}
I20260929 10:52:50.187939 3763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:50.187630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.193:53659)
0929 10:52:50.187822 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:50.187854 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":77,"threads_started":1}
I20260929 10:52:50.266803 3765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:50.266552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.195:48133)
0929 10:52:50.266726 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:50.266761 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":72,"threads_started":1}
I20260929 10:52:51.188732 3766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:51.188458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.193:35889)
0929 10:52:51.188615 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:51.188680 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":60,"threads_started":1}
I20260929 10:52:51.188735 3767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:51.188466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.194:34037)
0929 10:52:51.188623 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:51.188681 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":87,"threads_started":1}
I20260929 10:52:51.267530 3768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:51.267308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.195:45819)
0929 10:52:51.267450 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:51.267486 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":65,"threads_started":1}
I20260929 10:52:52.189443 3770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:52.189231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.193:38283)
0929 10:52:52.189358 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:52.189393 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
I20260929 10:52:52.189463 3769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:52.189229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.194:40725)
0929 10:52:52.189381 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:52.189422 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":74,"threads_started":1}
I20260929 10:52:52.268208 3772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:52.268009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.195:59795)
0929 10:52:52.268123 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:52.268160 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":58,"threads_started":1}
I20260929 10:52:53.190258 3773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:53.189997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.194:38393)
0929 10:52:53.190147 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:53.190186 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":61,"threads_started":1}
I20260929 10:52:53.190299 3774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:53.190028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.193:54157)
0929 10:52:53.190213 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:53.190250 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":108,"threads_started":1}
I20260929 10:52:53.262650 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 3290
I20260929 10:52:53.267540 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 3454
I20260929 10:52:53.269344 3775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:53.269139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37853 (local address 127.20.80.195:45491)
0929 10:52:53.269266 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:53.269304 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:37853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":61,"threads_started":1}
I20260929 10:52:53.271803 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 3589
2026-09-29T10:52:53Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18257 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:52:53 dist-test-slave-vqch krb5kdc[3778](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:52:53 dist-test-slave-vqch krb5kdc[3778](info): set up 2 sockets
Sep 29 10:52:53 dist-test-slave-vqch krb5kdc[3778](info): commencing operation
krb5kdc: starting...
W20260929 10:52:54.320945 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:52:54 dist-test-slave-vqch krb5kdc[3778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:52:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:52:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.80.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
I20260929 10:52:54.440325 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:39195
--webserver_interface=127.20.80.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:46069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:39195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:52:54.521037 3791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:52:54.521262 3791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:52:54.521297 3791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:52:54.522420 3791 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 10:52:54.522482 3791 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 10:52:54.522505 3791 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 10:52:54.522535 3791 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:52:54.522579 3791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:52:54.522612 3791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:52:54.522641 3791 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:52:54.522670 3791 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:52:54.524096 3791 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:46069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:39195
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:39195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:52:54.524364 3791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:52:54.524580 3791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:52:54.529553 3796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:54.529567 3797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:54.529660 3791 server_base.cc:1034] running on GCE node
W20260929 10:52:54.529618 3799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:54.530063 3791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:52:54.530313 3791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:52:54.531481 3791 hybrid_clock.cc:648] HybridClock initialized: now 1790679174531466 us; error 28 us; skew 500 ppm
Sep 29 10:52:54 dist-test-slave-vqch krb5kdc[3778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679174, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:52:54.533986 3791 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:52:54.534471 3791 webserver.cc:460] Webserver started at http://127.20.80.254:42113/ using document root <none> and password file <none>
I20260929 10:52:54.534655 3791 fs_manager.cc:359] Metadata directory not provided
I20260929 10:52:54.534695 3791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:52:54.534780 3791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:52:54.535756 3791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "a1657adc941042819338bbf51f53f537"
format_stamp: "Formatted at 2026-09-29 10:52:54 on dist-test-slave-vqch"
server_key: "baeaf904359347022e9a2a8d6e8ec9ad"
server_key_iv: "1577570795ac1af689bf3c04ebc2732e"
server_key_version: "encryptionkey@0"
I20260929 10:52:54.536162 3791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "a1657adc941042819338bbf51f53f537"
format_stamp: "Formatted at 2026-09-29 10:52:54 on dist-test-slave-vqch"
server_key: "baeaf904359347022e9a2a8d6e8ec9ad"
server_key_iv: "1577570795ac1af689bf3c04ebc2732e"
server_key_version: "encryptionkey@0"
I20260929 10:52:54.537613 3791 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:52:54.538384 3806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:54.538575 3791 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:52:54.538683 3791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "a1657adc941042819338bbf51f53f537"
format_stamp: "Formatted at 2026-09-29 10:52:54 on dist-test-slave-vqch"
server_key: "baeaf904359347022e9a2a8d6e8ec9ad"
server_key_iv: "1577570795ac1af689bf3c04ebc2732e"
server_key_version: "encryptionkey@0"
I20260929 10:52:54.538776 3791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:52:54.567327 3791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:52:54.569191 3791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:52:54.569358 3791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:52:54.573410 3791 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:39195
I20260929 10:52:54.573458 3858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:39195 every 8 connection(s)
I20260929 10:52:54.573844 3791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:52:54.574699 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:52:54.575001 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 3791
I20260929 10:52:54.575105 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:52:54.575313 20803 external_mini_cluster.cc:1428] Setting key 90c0d32e1fb96d2804b000a744a4e387
I20260929 10:52:54.578264 3859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537: Bootstrap starting.
I20260929 10:52:54.578958 3859 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537: Neither blocks nor log segments found. Creating new log.
I20260929 10:52:54.579267 3859 log.cc:824] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537: Log is configured to *not* fsync() on all Append() calls
I20260929 10:52:54.579983 3859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537: No bootstrap required, opened a new log
Sep 29 10:52:54 dist-test-slave-vqch krb5kdc[3778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:52:54.582620 3859 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1657adc941042819338bbf51f53f537" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 39195 } }
I20260929 10:52:54.582774 3859 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:52:54.582820 3859 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1657adc941042819338bbf51f53f537, State: Initialized, Role: FOLLOWER
I20260929 10:52:54.582991 3859 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1657adc941042819338bbf51f53f537" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 39195 } }
I20260929 10:52:54.583087 3859 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:52:54.583130 3859 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:52:54.583181 3859 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:52:54.583834 3859 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1657adc941042819338bbf51f53f537" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 39195 } }
I20260929 10:52:54.583953 3859 leader_election.cc:302] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1657adc941042819338bbf51f53f537; no voters:
I20260929 10:52:54.584124 3859 leader_election.cc:288] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:52:54.584173 3864 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:52:54.584326 3864 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [term 1 LEADER]: Becoming Leader. State: Replica: a1657adc941042819338bbf51f53f537, State: Running, Role: LEADER
I20260929 10:52:54.584408 3859 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:52:54.584544 3864 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1657adc941042819338bbf51f53f537" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 39195 } }
I20260929 10:52:54.585040 3866 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1657adc941042819338bbf51f53f537" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1657adc941042819338bbf51f53f537" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 39195 } } }
I20260929 10:52:54.585198 3866 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [sys.catalog]: This master's current role is: LEADER
I20260929 10:52:54.584908 3864 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1657adc941042819338bbf51f53f537. Latest consensus state: current_term: 1 leader_uuid: "a1657adc941042819338bbf51f53f537" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1657adc941042819338bbf51f53f537" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 39195 } } }
I20260929 10:52:54.585374 3864 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [sys.catalog]: This master's current role is: LEADER
W20260929 10:52:54.586265 3879 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:52:54.586338 3879 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:52:54.586570 3873 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:52:54.586738 3862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:54.575899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34786 (local address 127.20.80.254:39195)
0929 10:52:54.576053 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:54.576068 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:54.576094 (+ 26us) server_negotiation.cc:407] Connection header received
0929 10:52:54.576526 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:54.576532 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:54.576585 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:54.576708 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:54.577611 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:54.578619 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:54.579709 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:54.580164 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:54.580247 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:54.582936 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:54.582941 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:54.582942 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:54.582954 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:54.584936 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:54.585394 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:54.585397 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:54.585397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:54.585503 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:54.585726 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:54.585727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:54.585727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:54.586057 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:54.586283 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:54.586404 (+ 121us) server_negotiation.cc:299] Negotiation successful
0929 10:52:54.586541 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":79,"threads_started":1}
I20260929 10:52:54.586742 3873 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:52:54.588414 3873 catalog_manager.cc:1347] Generated new cluster ID: e51b1a7e7f214bc48fb8541e0ad3972e
I20260929 10:52:54.588467 3873 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:52:54.602032 3873 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:52:54.602582 3873 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:52:54.604393 3873 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537: Generated new TSK 0
I20260929 10:52:54.604569 3873 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.80.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
I20260929 10:52:54.635983 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.193:0
--local_ip_for_outbound_sockets=127.20.80.193
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:39195
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:46069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:52:54.717289 3885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:52:54.717515 3885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:52:54.717555 3885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:52:54.718713 3885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:52:54.718783 3885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:52:54.718832 3885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.193
I20260929 10:52:54.720355 3885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:46069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:39195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.193
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:52:54.720618 3885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:52:54.720856 3885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:52:54.725675 3890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:54.725688 3891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:54.725718 3893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:54.725926 3885 server_base.cc:1034] running on GCE node
I20260929 10:52:54.726186 3885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:52:54.726435 3885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:52:54.727581 3885 hybrid_clock.cc:648] HybridClock initialized: now 1790679174727566 us; error 26 us; skew 500 ppm
Sep 29 10:52:54 dist-test-slave-vqch krb5kdc[3778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679174, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:52:54.730126 3885 init.cc:377] Logged in from keytab as kudu/127.20.80.193@KRBTEST.COM (short username kudu)
I20260929 10:52:54.730575 3885 webserver.cc:460] Webserver started at http://127.20.80.193:36339/ using document root <none> and password file <none>
I20260929 10:52:54.730798 3885 fs_manager.cc:359] Metadata directory not provided
I20260929 10:52:54.730841 3885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:52:54.730924 3885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:52:54.731886 3885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/data/instance:
uuid: "6068e15b32f344179f1972d1fe7e3c52"
format_stamp: "Formatted at 2026-09-29 10:52:54 on dist-test-slave-vqch"
server_key: "a243feabe40acd714bdb8f24f725bb07"
server_key_iv: "e19c45bb6b848a94639cac5062fab8ae"
server_key_version: "encryptionkey@0"
I20260929 10:52:54.732165 3885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance:
uuid: "6068e15b32f344179f1972d1fe7e3c52"
format_stamp: "Formatted at 2026-09-29 10:52:54 on dist-test-slave-vqch"
server_key: "a243feabe40acd714bdb8f24f725bb07"
server_key_iv: "e19c45bb6b848a94639cac5062fab8ae"
server_key_version: "encryptionkey@0"
I20260929 10:52:54.733469 3885 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:52:54.734158 3900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:54.734338 3885 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:52:54.734414 3885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/wal
uuid: "6068e15b32f344179f1972d1fe7e3c52"
format_stamp: "Formatted at 2026-09-29 10:52:54 on dist-test-slave-vqch"
server_key: "a243feabe40acd714bdb8f24f725bb07"
server_key_iv: "e19c45bb6b848a94639cac5062fab8ae"
server_key_version: "encryptionkey@0"
I20260929 10:52:54.734464 3885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:52:54.782636 3885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:52:54.785084 3885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:52:54.785240 3885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:52:54.785481 3885 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:52:54.785764 3885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:52:54.785799 3885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:54.785823 3885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:52:54.785840 3885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:54.791637 3885 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.193:44365
I20260929 10:52:54.791684 4014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.193:44365 every 8 connection(s)
I20260929 10:52:54.792018 3885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
Sep 29 10:52:54 dist-test-slave-vqch krb5kdc[3778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679174, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:52:54.800946 3862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:54.792720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:54573 (local address 127.20.80.254:39195)
0929 10:52:54.792792 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:54.792794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:54.793045 (+ 251us) server_negotiation.cc:407] Connection header received
0929 10:52:54.793564 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:54.793565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:54.793575 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:54.793629 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:54.794552 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:54.795058 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:54.796074 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:54.796271 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:54.796320 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:54.798498 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:54.798499 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:54.798499 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:54.798506 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:54.799783 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:54.800167 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:54.800167 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:54.800168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:54.800217 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:54.800428 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:54.800428 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:54.800429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:54.800527 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:54.800577 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:54.800810 (+ 233us) server_negotiation.cc:299] Negotiation successful
0929 10:52:54.800879 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 10:52:54.801216 4017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:54.792712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.193:54573)
0929 10:52:54.792899 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:54.792938 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:54.793476 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:54.793705 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:54.793709 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:54.794278 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:52:54.794473 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:54.794478 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:54.795166 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:54.795167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:54.795994 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:54.795996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:54.796345 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:54.796346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:54.796466 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:54.796938 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:52:54.796947 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:52:54.798407 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:52:54.799871 (+ 1464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:54.799878 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:54.799880 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:54.800096 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:54.800274 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:54.800275 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:54.800275 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:54.800372 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:54.800581 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:52:54.800586 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:52:54.800703 (+ 117us) client_negotiation.cc:769] Sending connection context
0929 10:52:54.800786 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 10:52:54.800906 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":107,"threads_started":1}
I20260929 10:52:54.801496 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 3885
I20260929 10:52:54.801600 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance
I20260929 10:52:54.801834 20803 external_mini_cluster.cc:1428] Setting key 8869d481ce20e75b61f1a50edd0f912d
I20260929 10:52:54.801932 4015 heartbeater.cc:347] Connected to a master server at 127.20.80.254:39195
I20260929 10:52:54.802050 4015 heartbeater.cc:464] Registering TS with master...
I20260929 10:52:54.802242 4015 heartbeater.cc:511] Master 127.20.80.254:39195 requested a full tablet report, sending...
I20260929 10:52:54.802971 3823 ts_manager.cc:194] Registered new tserver with Master: 6068e15b32f344179f1972d1fe7e3c52 (127.20.80.193:44365)
I20260929 10:52:54.803866 3823 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.193@KRBTEST.COM'} at 127.20.80.193:54573
WARNING: no policy specified for kudu/127.20.80.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
I20260929 10:52:54.827538 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.194:0
--local_ip_for_outbound_sockets=127.20.80.194
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:39195
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:46069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:52:54.909066 4020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:52:54.909271 4020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:52:54.909294 4020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:52:54.910374 4020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:52:54.910409 4020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:52:54.910434 4020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.194
I20260929 10:52:54.911801 4020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:46069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:39195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.194
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:52:54.911985 4020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:52:54.912245 4020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:52:54.917176 4026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:54.917203 4020 server_base.cc:1034] running on GCE node
W20260929 10:52:54.917240 4025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:54.917176 4028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:54.917614 4020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:52:54.917886 4020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:52:54.919037 4020 hybrid_clock.cc:648] HybridClock initialized: now 1790679174919019 us; error 40 us; skew 500 ppm
Sep 29 10:52:54 dist-test-slave-vqch krb5kdc[3778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679174, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:52:54.925550 4020 init.cc:377] Logged in from keytab as kudu/127.20.80.194@KRBTEST.COM (short username kudu)
I20260929 10:52:54.925992 4020 webserver.cc:460] Webserver started at http://127.20.80.194:45783/ using document root <none> and password file <none>
I20260929 10:52:54.926225 4020 fs_manager.cc:359] Metadata directory not provided
I20260929 10:52:54.926311 4020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:52:54.926460 4020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:52:54.927503 4020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/data/instance:
uuid: "8ba6deb8d47147268bd65549332255ba"
format_stamp: "Formatted at 2026-09-29 10:52:54 on dist-test-slave-vqch"
server_key: "b099fc61df5a00b45a75c1142b1fb82f"
server_key_iv: "748e729b9178b339e4e0e5ef01d61646"
server_key_version: "encryptionkey@0"
I20260929 10:52:54.927858 4020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance:
uuid: "8ba6deb8d47147268bd65549332255ba"
format_stamp: "Formatted at 2026-09-29 10:52:54 on dist-test-slave-vqch"
server_key: "b099fc61df5a00b45a75c1142b1fb82f"
server_key_iv: "748e729b9178b339e4e0e5ef01d61646"
server_key_version: "encryptionkey@0"
I20260929 10:52:54.929252 4020 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:52:54.929970 4035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:54.930188 4020 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:52:54.930311 4020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/wal
uuid: "8ba6deb8d47147268bd65549332255ba"
format_stamp: "Formatted at 2026-09-29 10:52:54 on dist-test-slave-vqch"
server_key: "b099fc61df5a00b45a75c1142b1fb82f"
server_key_iv: "748e729b9178b339e4e0e5ef01d61646"
server_key_version: "encryptionkey@0"
I20260929 10:52:54.930393 4020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:52:54.939603 4020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:52:54.942186 4020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:52:54.942359 4020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:52:54.942606 4020 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:52:54.942925 4020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:52:54.942983 4020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:54.943028 4020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:52:54.943073 4020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:54.949517 4020 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.194:39931
I20260929 10:52:54.949561 4149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.194:39931 every 8 connection(s)
I20260929 10:52:54.949919 4020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
I20260929 10:52:54.952804 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 4020
I20260929 10:52:54.952895 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance
I20260929 10:52:54.953130 20803 external_mini_cluster.cc:1428] Setting key 9ab3d64bf5702a9e705feb3e01359205
Sep 29 10:52:54 dist-test-slave-vqch krb5kdc[3778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679174, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:52:54.960558 3862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:54.950747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:47939 (local address 127.20.80.254:39195)
0929 10:52:54.950816 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:54.950818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:54.951024 (+ 206us) server_negotiation.cc:407] Connection header received
0929 10:52:54.951508 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:54.951509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:54.951518 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:54.951577 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:54.952516 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:54.953149 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:54.954458 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:54.954644 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:54.954699 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:54.957321 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:54.957323 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:54.957323 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:54.957337 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:54.959076 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:54.959514 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:54.959515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:54.959516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:54.959581 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:54.959849 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:54.959850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:54.959851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:54.959995 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:54.960078 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:54.960350 (+ 272us) server_negotiation.cc:299] Negotiation successful
0929 10:52:54.960451 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 10:52:54.960580 4152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:54.950730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.194:47939)
0929 10:52:54.950889 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:54.950931 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:54.951424 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:54.951632 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:54.951635 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:54.952215 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:52:54.952429 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:54.952434 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:54.953292 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:54.953293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:54.954364 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:54.954368 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:54.954731 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:54.954732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:54.954879 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:54.955514 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:52:54.955522 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:52:54.957226 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:52:54.959178 (+ 1952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:54.959185 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:54.959188 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:54.959431 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:54.959658 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:54.959659 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:54.959659 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:54.959774 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:54.960071 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:52:54.960076 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:52:54.960205 (+ 129us) client_negotiation.cc:769] Sending connection context
0929 10:52:54.960296 (+ 91us) client_negotiation.cc:240] Negotiation successful
0929 10:52:54.960420 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":84,"threads_started":1}
I20260929 10:52:54.961268 4150 heartbeater.cc:347] Connected to a master server at 127.20.80.254:39195
I20260929 10:52:54.961377 4150 heartbeater.cc:464] Registering TS with master...
I20260929 10:52:54.961571 4150 heartbeater.cc:511] Master 127.20.80.254:39195 requested a full tablet report, sending...
I20260929 10:52:54.962000 3823 ts_manager.cc:194] Registered new tserver with Master: 8ba6deb8d47147268bd65549332255ba (127.20.80.194:39931)
WARNING: no policy specified for kudu/127.20.80.195@KRBTEST.COM; defaulting to no policy
I20260929 10:52:54.962430 3823 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.194@KRBTEST.COM'} at 127.20.80.194:47939
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
I20260929 10:52:54.979727 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.195:0
--local_ip_for_outbound_sockets=127.20.80.195
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:39195
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:46069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:52:55.061611 4155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:52:55.061817 4155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:52:55.061843 4155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:52:55.063002 4155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:52:55.063045 4155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:52:55.063072 4155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.195
I20260929 10:52:55.064541 4155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:46069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:39195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.195
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:52:55.064805 4155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:52:55.065097 4155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:52:55.070251 4163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:52:55.070292 4161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:55.070497 4155 server_base.cc:1034] running on GCE node
W20260929 10:52:55.070308 4160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:52:55.070827 4155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:52:55.071074 4155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:52:55.072237 4155 hybrid_clock.cc:648] HybridClock initialized: now 1790679175072224 us; error 52 us; skew 500 ppm
Sep 29 10:52:55 dist-test-slave-vqch krb5kdc[3778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679175, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:52:55.074893 4155 init.cc:377] Logged in from keytab as kudu/127.20.80.195@KRBTEST.COM (short username kudu)
I20260929 10:52:55.075318 4155 webserver.cc:460] Webserver started at http://127.20.80.195:39871/ using document root <none> and password file <none>
I20260929 10:52:55.075559 4155 fs_manager.cc:359] Metadata directory not provided
I20260929 10:52:55.075629 4155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:52:55.075790 4155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:52:55.076835 4155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/data/instance:
uuid: "ee54fe9fec6d4197862fa22f8a07fa35"
format_stamp: "Formatted at 2026-09-29 10:52:55 on dist-test-slave-vqch"
server_key: "9c6042a48842f0a37e6165c171fd5bf8"
server_key_iv: "93102fa2138c42adf8f2c705798eb20b"
server_key_version: "encryptionkey@0"
I20260929 10:52:55.077219 4155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance:
uuid: "ee54fe9fec6d4197862fa22f8a07fa35"
format_stamp: "Formatted at 2026-09-29 10:52:55 on dist-test-slave-vqch"
server_key: "9c6042a48842f0a37e6165c171fd5bf8"
server_key_iv: "93102fa2138c42adf8f2c705798eb20b"
server_key_version: "encryptionkey@0"
I20260929 10:52:55.078524 4155 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:52:55.079244 4170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:55.079416 4155 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:52:55.079514 4155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/wal
uuid: "ee54fe9fec6d4197862fa22f8a07fa35"
format_stamp: "Formatted at 2026-09-29 10:52:55 on dist-test-slave-vqch"
server_key: "9c6042a48842f0a37e6165c171fd5bf8"
server_key_iv: "93102fa2138c42adf8f2c705798eb20b"
server_key_version: "encryptionkey@0"
I20260929 10:52:55.079592 4155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:52:55.087641 4155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:52:55.089985 4155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:52:55.090148 4155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:52:55.090366 4155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:52:55.090672 4155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:52:55.090727 4155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:55.090770 4155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:52:55.090812 4155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:52:55.097079 4155 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.195:39941
I20260929 10:52:55.097131 4283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.195:39941 every 8 connection(s)
I20260929 10:52:55.097481 4155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
Sep 29 10:52:55 dist-test-slave-vqch krb5kdc[3778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679175, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:52:55.105157 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 4155
I20260929 10:52:55.105227 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance
I20260929 10:52:55.105406 20803 external_mini_cluster.cc:1428] Setting key b64a688ea268da89544b4feb5bd771d2
I20260929 10:52:55.106868 3862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:55.098183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:60203 (local address 127.20.80.254:39195)
0929 10:52:55.098254 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:55.098261 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:55.098488 (+ 227us) server_negotiation.cc:407] Connection header received
0929 10:52:55.098944 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:55.098945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:55.098954 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:55.099012 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:55.099930 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:55.100437 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:55.101485 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:55.101668 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:55.101723 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:55.103718 (+ 1995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:55.103720 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:55.103720 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:55.103727 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:55.105383 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:55.105830 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:55.105831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:55.105832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:55.105894 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:55.106184 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:55.106185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:55.106185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:55.106355 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:55.106411 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:55.106695 (+ 284us) server_negotiation.cc:299] Negotiation successful
0929 10:52:55.106778 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 10:52:55.107064 4286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:55.098175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.195:60203)
0929 10:52:55.098353 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:55.098396 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:55.098860 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:55.099063 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:55.099066 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:55.099626 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:52:55.099823 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:55.099828 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:55.100520 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:55.100521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:55.101410 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:55.101413 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:55.101743 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:55.101744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:55.101849 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:55.102289 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:52:55.102297 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:52:55.103634 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:52:55.105460 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:55.105469 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:55.105471 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:55.105739 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:55.105952 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:52:55.105953 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:52:55.105954 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:52:55.106119 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:52:55.106416 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:52:55.106422 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:52:55.106558 (+ 136us) client_negotiation.cc:769] Sending connection context
0929 10:52:55.106662 (+ 104us) client_negotiation.cc:240] Negotiation successful
0929 10:52:55.106829 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":84,"threads_started":1}
I20260929 10:52:55.107510 4284 heartbeater.cc:347] Connected to a master server at 127.20.80.254:39195
I20260929 10:52:55.107589 4284 heartbeater.cc:464] Registering TS with master...
I20260929 10:52:55.107722 4284 heartbeater.cc:511] Master 127.20.80.254:39195 requested a full tablet report, sending...
I20260929 10:52:55.108039 3823 ts_manager.cc:194] Registered new tserver with Master: ee54fe9fec6d4197862fa22f8a07fa35 (127.20.80.195:39941)
I20260929 10:52:55.108419 3823 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.195@KRBTEST.COM'} at 127.20.80.195:60203
I20260929 10:52:55.117319 20803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:52:55.124302 3862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:55.118688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34792 (local address 127.20.80.254:39195)
0929 10:52:55.118729 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:55.118733 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:55.118818 (+ 85us) server_negotiation.cc:407] Connection header received
0929 10:52:55.118872 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:55.118873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:55.118881 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:55.118933 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:52:55.119427 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:55.119907 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:55.120599 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:55.120801 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:55.120856 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:55.122107 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:52:55.122108 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:52:55.122108 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:52:55.122114 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:52:55.123304 (+ 1190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:55.123630 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:55.123631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:55.123632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:55.123687 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:52:55.123877 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:52:55.123878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:52:55.123878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:52:55.124017 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:52:55.124094 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:55.124180 (+ 86us) server_negotiation.cc:299] Negotiation successful
0929 10:52:55.124239 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 10:52:55.126044 3823 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34792:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:52:55.127619 3823 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:52:55.134094 4295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:55.129921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:39941 (local address 127.0.0.1:36364)
0929 10:52:55.130081 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:55.130116 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:55.130181 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:55.130533 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:55.130536 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:55.130538 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:52:55.130684 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:55.130687 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:55.132030 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:55.132031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:55.132867 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:55.132869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:55.133794 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:55.133796 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:55.133883 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:55.133896 (+ 13us) client_negotiation.cc:769] Sending connection context
0929 10:52:55.133970 (+ 74us) client_negotiation.cc:240] Negotiation successful
0929 10:52:55.134017 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":34,"thread_start_us":70,"threads_started":1}
I20260929 10:52:55.134145 4299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:55.130180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36364 (local address 127.20.80.195:39941)
0929 10:52:55.130370 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:55.130376 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:55.130387 (+ 11us) server_negotiation.cc:407] Connection header received
0929 10:52:55.130427 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:55.130430 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:55.130447 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:55.130528 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:52:55.130771 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:55.131905 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:55.132962 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:55.133709 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:55.133787 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:55.133849 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:55.134001 (+ 152us) server_negotiation.cc:299] Negotiation successful
0929 10:52:55.134045 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":92,"threads_started":1}
I20260929 10:52:55.134232 4296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:55.129921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:44365 (local address 127.0.0.1:41722)
0929 10:52:55.130353 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:55.130363 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:55.130423 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:55.130588 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:55.130589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:55.130591 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:52:55.130692 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:55.130695 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:55.131764 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:55.131765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:55.133015 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:55.133017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:55.133994 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:55.133996 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:55.134056 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:55.134060 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:52:55.134111 (+ 51us) client_negotiation.cc:240] Negotiation successful
0929 10:52:55.134153 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"mutex_wait_us":40,"thread_start_us":56,"threads_started":1}
I20260929 10:52:55.134718 4297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:55.129940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:39931 (local address 127.0.0.1:38882)
0929 10:52:55.130365 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:55.130375 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:55.130452 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:55.130797 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:55.130798 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:55.130799 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:52:55.130853 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:55.130855 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:55.132054 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:55.132055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:55.133533 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:55.133536 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:55.134505 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:55.134506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:55.134577 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:55.134580 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:52:55.134605 (+ 25us) client_negotiation.cc:240] Negotiation successful
0929 10:52:55.134632 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"mutex_wait_us":35,"thread_start_us":93,"threads_started":1}
I20260929 10:52:55.134442 4298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:55.130106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41722 (local address 127.20.80.193:44365)
0929 10:52:55.130299 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:55.130303 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:55.130438 (+ 135us) server_negotiation.cc:407] Connection header received
0929 10:52:55.130491 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:55.130494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:55.130510 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:55.130581 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:52:55.130931 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:55.131661 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:55.133082 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:55.133835 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:55.133896 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:55.133939 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:55.134294 (+ 355us) server_negotiation.cc:299] Negotiation successful
0929 10:52:55.134342 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":83,"threads_started":1}
I20260929 10:52:55.135109 4301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:55.130181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38882 (local address 127.20.80.194:39931)
0929 10:52:55.130659 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:55.130663 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:55.130672 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:52:55.130710 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:55.130711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:55.130727 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:55.130804 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:52:55.130911 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:55.131954 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:55.133610 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:55.134393 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:55.134873 (+ 480us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:55.134947 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:55.134973 (+ 26us) server_negotiation.cc:299] Negotiation successful
0929 10:52:55.135022 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":95,"threads_started":1}
I20260929 10:52:55.135200 4218 tablet_service.cc:1467] Processing CreateTablet for tablet 79b47648b6f046438dafa78af273da05 (DEFAULT_TABLE table=test-table [id=5599160681454b819ce5a812324148fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:52:55.135296 3949 tablet_service.cc:1467] Processing CreateTablet for tablet 79b47648b6f046438dafa78af273da05 (DEFAULT_TABLE table=test-table [id=5599160681454b819ce5a812324148fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:52:55.135586 4218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79b47648b6f046438dafa78af273da05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:52:55.135586 3949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79b47648b6f046438dafa78af273da05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:52:55.135713 4084 tablet_service.cc:1467] Processing CreateTablet for tablet 79b47648b6f046438dafa78af273da05 (DEFAULT_TABLE table=test-table [id=5599160681454b819ce5a812324148fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:52:55.136039 4084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79b47648b6f046438dafa78af273da05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:52:55.138959 4302 tablet_bootstrap.cc:492] T 79b47648b6f046438dafa78af273da05 P 8ba6deb8d47147268bd65549332255ba: Bootstrap starting.
I20260929 10:52:55.139299 4304 tablet_bootstrap.cc:492] T 79b47648b6f046438dafa78af273da05 P 6068e15b32f344179f1972d1fe7e3c52: Bootstrap starting.
I20260929 10:52:55.139303 4303 tablet_bootstrap.cc:492] T 79b47648b6f046438dafa78af273da05 P ee54fe9fec6d4197862fa22f8a07fa35: Bootstrap starting.
I20260929 10:52:55.139650 4302 tablet_bootstrap.cc:654] T 79b47648b6f046438dafa78af273da05 P 8ba6deb8d47147268bd65549332255ba: Neither blocks nor log segments found. Creating new log.
I20260929 10:52:55.139951 4303 tablet_bootstrap.cc:654] T 79b47648b6f046438dafa78af273da05 P ee54fe9fec6d4197862fa22f8a07fa35: Neither blocks nor log segments found. Creating new log.
I20260929 10:52:55.139951 4304 tablet_bootstrap.cc:654] T 79b47648b6f046438dafa78af273da05 P 6068e15b32f344179f1972d1fe7e3c52: Neither blocks nor log segments found. Creating new log.
I20260929 10:52:55.140199 4302 log.cc:824] T 79b47648b6f046438dafa78af273da05 P 8ba6deb8d47147268bd65549332255ba: Log is configured to *not* fsync() on all Append() calls
I20260929 10:52:55.140576 4304 log.cc:824] T 79b47648b6f046438dafa78af273da05 P 6068e15b32f344179f1972d1fe7e3c52: Log is configured to *not* fsync() on all Append() calls
I20260929 10:52:55.140576 4303 log.cc:824] T 79b47648b6f046438dafa78af273da05 P ee54fe9fec6d4197862fa22f8a07fa35: Log is configured to *not* fsync() on all Append() calls
I20260929 10:52:55.140945 4302 tablet_bootstrap.cc:492] T 79b47648b6f046438dafa78af273da05 P 8ba6deb8d47147268bd65549332255ba: No bootstrap required, opened a new log
I20260929 10:52:55.141053 4302 ts_tablet_manager.cc:1397] T 79b47648b6f046438dafa78af273da05 P 8ba6deb8d47147268bd65549332255ba: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:52:55.141423 4303 tablet_bootstrap.cc:492] T 79b47648b6f046438dafa78af273da05 P ee54fe9fec6d4197862fa22f8a07fa35: No bootstrap required, opened a new log
I20260929 10:52:55.141492 4303 ts_tablet_manager.cc:1397] T 79b47648b6f046438dafa78af273da05 P ee54fe9fec6d4197862fa22f8a07fa35: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:52:55.141561 4304 tablet_bootstrap.cc:492] T 79b47648b6f046438dafa78af273da05 P 6068e15b32f344179f1972d1fe7e3c52: No bootstrap required, opened a new log
I20260929 10:52:55.141623 4304 ts_tablet_manager.cc:1397] T 79b47648b6f046438dafa78af273da05 P 6068e15b32f344179f1972d1fe7e3c52: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:52:55.142803 4302 raft_consensus.cc:348] T 79b47648b6f046438dafa78af273da05 P 8ba6deb8d47147268bd65549332255ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6068e15b32f344179f1972d1fe7e3c52" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 44365 } } peers { permanent_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39941 } } peers { permanent_uuid: "8ba6deb8d47147268bd65549332255ba" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 39931 } }
I20260929 10:52:55.143002 4302 raft_consensus.cc:374] T 79b47648b6f046438dafa78af273da05 P 8ba6deb8d47147268bd65549332255ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:52:55.143045 4302 raft_consensus.cc:729] T 79b47648b6f046438dafa78af273da05 P 8ba6deb8d47147268bd65549332255ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ba6deb8d47147268bd65549332255ba, State: Initialized, Role: FOLLOWER
I20260929 10:52:55.143163 4302 consensus_queue.cc:260] T 79b47648b6f046438dafa78af273da05 P 8ba6deb8d47147268bd65549332255ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6068e15b32f344179f1972d1fe7e3c52" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 44365 } } peers { permanent_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39941 } } peers { permanent_uuid: "8ba6deb8d47147268bd65549332255ba" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 39931 } }
I20260929 10:52:55.143451 4302 ts_tablet_manager.cc:1428] T 79b47648b6f046438dafa78af273da05 P 8ba6deb8d47147268bd65549332255ba: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:52:55.143587 4150 heartbeater.cc:503] Master 127.20.80.254:39195 was elected leader, sending a full tablet report...
I20260929 10:52:55.143539 4304 raft_consensus.cc:348] T 79b47648b6f046438dafa78af273da05 P 6068e15b32f344179f1972d1fe7e3c52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6068e15b32f344179f1972d1fe7e3c52" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 44365 } } peers { permanent_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39941 } } peers { permanent_uuid: "8ba6deb8d47147268bd65549332255ba" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 39931 } }
I20260929 10:52:55.143539 4303 raft_consensus.cc:348] T 79b47648b6f046438dafa78af273da05 P ee54fe9fec6d4197862fa22f8a07fa35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6068e15b32f344179f1972d1fe7e3c52" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 44365 } } peers { permanent_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39941 } } peers { permanent_uuid: "8ba6deb8d47147268bd65549332255ba" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 39931 } }
I20260929 10:52:55.143714 4303 raft_consensus.cc:374] T 79b47648b6f046438dafa78af273da05 P ee54fe9fec6d4197862fa22f8a07fa35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:52:55.143714 4304 raft_consensus.cc:374] T 79b47648b6f046438dafa78af273da05 P 6068e15b32f344179f1972d1fe7e3c52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:52:55.143759 4303 raft_consensus.cc:729] T 79b47648b6f046438dafa78af273da05 P ee54fe9fec6d4197862fa22f8a07fa35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee54fe9fec6d4197862fa22f8a07fa35, State: Initialized, Role: FOLLOWER
I20260929 10:52:55.143759 4304 raft_consensus.cc:729] T 79b47648b6f046438dafa78af273da05 P 6068e15b32f344179f1972d1fe7e3c52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6068e15b32f344179f1972d1fe7e3c52, State: Initialized, Role: FOLLOWER
I20260929 10:52:55.143868 4304 consensus_queue.cc:260] T 79b47648b6f046438dafa78af273da05 P 6068e15b32f344179f1972d1fe7e3c52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6068e15b32f344179f1972d1fe7e3c52" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 44365 } } peers { permanent_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39941 } } peers { permanent_uuid: "8ba6deb8d47147268bd65549332255ba" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 39931 } }
I20260929 10:52:55.143865 4303 consensus_queue.cc:260] T 79b47648b6f046438dafa78af273da05 P ee54fe9fec6d4197862fa22f8a07fa35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6068e15b32f344179f1972d1fe7e3c52" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 44365 } } peers { permanent_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39941 } } peers { permanent_uuid: "8ba6deb8d47147268bd65549332255ba" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 39931 } }
I20260929 10:52:55.144131 4304 ts_tablet_manager.cc:1428] T 79b47648b6f046438dafa78af273da05 P 6068e15b32f344179f1972d1fe7e3c52: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 10:52:55.144131 4303 ts_tablet_manager.cc:1428] T 79b47648b6f046438dafa78af273da05 P ee54fe9fec6d4197862fa22f8a07fa35: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 10:52:55.144254 4284 heartbeater.cc:503] Master 127.20.80.254:39195 was elected leader, sending a full tablet report...
I20260929 10:52:55.144265 4015 heartbeater.cc:503] Master 127.20.80.254:39195 was elected leader, sending a full tablet report...
I20260929 10:52:55.167487 4308 raft_consensus.cc:482] T 79b47648b6f046438dafa78af273da05 P 8ba6deb8d47147268bd65549332255ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:52:55.167603 4308 raft_consensus.cc:504] T 79b47648b6f046438dafa78af273da05 P 8ba6deb8d47147268bd65549332255ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6068e15b32f344179f1972d1fe7e3c52" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 44365 } } peers { permanent_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39941 } } peers { permanent_uuid: "8ba6deb8d47147268bd65549332255ba" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 39931 } }
I20260929 10:52:55.167964 4308 leader_election.cc:288] T 79b47648b6f046438dafa78af273da05 P 8ba6deb8d47147268bd65549332255ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6068e15b32f344179f1972d1fe7e3c52 (127.20.80.193:44365), ee54fe9fec6d4197862fa22f8a07fa35 (127.20.80.195:39941)
I20260929 10:52:55.171868 4298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:55.168207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:50459 (local address 127.20.80.193:44365)
0929 10:52:55.168253 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:55.168255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:55.168268 (+ 13us) server_negotiation.cc:407] Connection header received
0929 10:52:55.168301 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:55.168301 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:55.168314 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:55.168372 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:52:55.168587 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:55.169367 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:55.170933 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:55.171508 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:55.171587 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:55.171596 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:55.171757 (+ 161us) server_negotiation.cc:299] Negotiation successful
0929 10:52:55.171810 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 10:52:55.171864 4299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:55.168209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:34401 (local address 127.20.80.195:39941)
0929 10:52:55.168263 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:55.168265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:55.168415 (+ 150us) server_negotiation.cc:407] Connection header received
0929 10:52:55.168448 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:55.168449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:55.168465 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:55.168506 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:52:55.168662 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:55.169377 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:55.170930 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:55.171510 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:55.171587 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:55.171596 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:55.171752 (+ 156us) server_negotiation.cc:299] Negotiation successful
0929 10:52:55.171793 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260929 10:52:55.171880 4152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:55.168133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:44365 (local address 127.20.80.194:50459)
0929 10:52:55.168193 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:55.168205 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:55.168245 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:55.168394 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:55.168395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:55.168397 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:52:55.168485 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:55.168486 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:55.169452 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:55.169453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:55.170817 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:55.170820 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:55.171593 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:55.171594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:55.171675 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:55.171676 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 10:52:55.171733 (+ 57us) client_negotiation.cc:240] Negotiation successful
0929 10:52:55.171781 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":84}
I20260929 10:52:55.171887 4311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:55.168159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:39941 (local address 127.20.80.194:34401)
0929 10:52:55.168334 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:55.168343 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:52:55.168400 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:52:55.168526 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:52:55.168527 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:52:55.168528 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:52:55.168596 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:55.168597 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:55.169452 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:55.169453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:55.170817 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:52:55.170820 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:52:55.171594 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:52:55.171595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:52:55.171672 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:55.171676 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:52:55.171733 (+ 57us) client_negotiation.cc:240] Negotiation successful
0929 10:52:55.171780 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":77,"thread_start_us":115,"threads_started":1}
I20260929 10:52:55.172245 3969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79b47648b6f046438dafa78af273da05" candidate_uuid: "8ba6deb8d47147268bd65549332255ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6068e15b32f344179f1972d1fe7e3c52" is_pre_election: true
I20260929 10:52:55.172411 3969 raft_consensus.cc:2454] T 79b47648b6f046438dafa78af273da05 P 6068e15b32f344179f1972d1fe7e3c52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ba6deb8d47147268bd65549332255ba in term 0.
I20260929 10:52:55.172402 4238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79b47648b6f046438dafa78af273da05" candidate_uuid: "8ba6deb8d47147268bd65549332255ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" is_pre_election: true
I20260929 10:52:55.172591 4238 raft_consensus.cc:2454] T 79b47648b6f046438dafa78af273da05 P ee54fe9fec6d4197862fa22f8a07fa35 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ba6deb8d47147268bd65549332255ba in term 0.
I20260929 10:52:55.172700 4036 leader_election.cc:302] T 79b47648b6f046438dafa78af273da05 P 8ba6deb8d47147268bd65549332255ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6068e15b32f344179f1972d1fe7e3c52, 8ba6deb8d47147268bd65549332255ba; no voters:
I20260929 10:52:55.172819 4308 raft_consensus.cc:2781] T 79b47648b6f046438dafa78af273da05 P 8ba6deb8d47147268bd65549332255ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:52:55.172887 4308 raft_consensus.cc:482] T 79b47648b6f046438dafa78af273da05 P 8ba6deb8d47147268bd65549332255ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:52:55.172904 4308 raft_consensus.cc:3037] T 79b47648b6f046438dafa78af273da05 P 8ba6deb8d47147268bd65549332255ba [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:52:55.173592 4308 raft_consensus.cc:504] T 79b47648b6f046438dafa78af273da05 P 8ba6deb8d47147268bd65549332255ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6068e15b32f344179f1972d1fe7e3c52" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 44365 } } peers { permanent_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39941 } } peers { permanent_uuid: "8ba6deb8d47147268bd65549332255ba" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 39931 } }
I20260929 10:52:55.173730 4308 leader_election.cc:288] T 79b47648b6f046438dafa78af273da05 P 8ba6deb8d47147268bd65549332255ba [CANDIDATE]: Term 1 election: Requested vote from peers 6068e15b32f344179f1972d1fe7e3c52 (127.20.80.193:44365), ee54fe9fec6d4197862fa22f8a07fa35 (127.20.80.195:39941)
I20260929 10:52:55.173915 3969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79b47648b6f046438dafa78af273da05" candidate_uuid: "8ba6deb8d47147268bd65549332255ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6068e15b32f344179f1972d1fe7e3c52"
I20260929 10:52:55.173980 3969 raft_consensus.cc:3037] T 79b47648b6f046438dafa78af273da05 P 6068e15b32f344179f1972d1fe7e3c52 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:52:55.174659 3969 raft_consensus.cc:2454] T 79b47648b6f046438dafa78af273da05 P 6068e15b32f344179f1972d1fe7e3c52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ba6deb8d47147268bd65549332255ba in term 1.
I20260929 10:52:55.174806 4238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79b47648b6f046438dafa78af273da05" candidate_uuid: "8ba6deb8d47147268bd65549332255ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee54fe9fec6d4197862fa22f8a07fa35"
I20260929 10:52:55.174857 4036 leader_election.cc:302] T 79b47648b6f046438dafa78af273da05 P 8ba6deb8d47147268bd65549332255ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6068e15b32f344179f1972d1fe7e3c52, 8ba6deb8d47147268bd65549332255ba; no voters:
I20260929 10:52:55.174898 4238 raft_consensus.cc:3037] T 79b47648b6f046438dafa78af273da05 P ee54fe9fec6d4197862fa22f8a07fa35 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:52:55.174955 4308 raft_consensus.cc:2781] T 79b47648b6f046438dafa78af273da05 P 8ba6deb8d47147268bd65549332255ba [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:52:55.175118 4308 raft_consensus.cc:686] T 79b47648b6f046438dafa78af273da05 P 8ba6deb8d47147268bd65549332255ba [term 1 LEADER]: Becoming Leader. State: Replica: 8ba6deb8d47147268bd65549332255ba, State: Running, Role: LEADER
I20260929 10:52:55.175220 4308 consensus_queue.cc:237] T 79b47648b6f046438dafa78af273da05 P 8ba6deb8d47147268bd65549332255ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6068e15b32f344179f1972d1fe7e3c52" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 44365 } } peers { permanent_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39941 } } peers { permanent_uuid: "8ba6deb8d47147268bd65549332255ba" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 39931 } }
I20260929 10:52:55.175812 4238 raft_consensus.cc:2454] T 79b47648b6f046438dafa78af273da05 P ee54fe9fec6d4197862fa22f8a07fa35 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ba6deb8d47147268bd65549332255ba in term 1.
I20260929 10:52:55.176074 3821 catalog_manager.cc:5495] T 79b47648b6f046438dafa78af273da05 P 8ba6deb8d47147268bd65549332255ba reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ba6deb8d47147268bd65549332255ba (127.20.80.194). New cstate: current_term: 1 leader_uuid: "8ba6deb8d47147268bd65549332255ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6068e15b32f344179f1972d1fe7e3c52" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 44365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ba6deb8d47147268bd65549332255ba" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 39931 } health_report { overall_health: HEALTHY } } }
I20260929 10:52:55.191237 4301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:55.188849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38894 (local address 127.20.80.194:39931)
0929 10:52:55.188907 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 10:52:55.188909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:52:55.188917 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:52:55.188953 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:52:55.188954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:52:55.188999 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:52:55.189066 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:52:55.189202 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:55.189669 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:55.190442 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:52:55.190640 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:52:55.190709 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:52:55.190889 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:52:55.190989 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:52:55.191085 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 10:52:55.191110 (+ 25us) server_negotiation.cc:299] Negotiation successful
0929 10:52:55.191130 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 10:52:55.192968 3969 raft_consensus.cc:1261] T 79b47648b6f046438dafa78af273da05 P 6068e15b32f344179f1972d1fe7e3c52 [term 1 FOLLOWER]: Refusing update from remote peer 8ba6deb8d47147268bd65549332255ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:52:55.192968 4238 raft_consensus.cc:1261] T 79b47648b6f046438dafa78af273da05 P ee54fe9fec6d4197862fa22f8a07fa35 [term 1 FOLLOWER]: Refusing update from remote peer 8ba6deb8d47147268bd65549332255ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:52:55.193337 4308 consensus_queue.cc:1035] T 79b47648b6f046438dafa78af273da05 P 8ba6deb8d47147268bd65549332255ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:52:55.193522 4312 consensus_queue.cc:1035] T 79b47648b6f046438dafa78af273da05 P 8ba6deb8d47147268bd65549332255ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "6068e15b32f344179f1972d1fe7e3c52" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 44365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:52:55.195808 4315 mvcc.cc:205] Tried to move back new op lower bound from 7334621901587566592 to 7334621901518286848. Current Snapshot: MvccSnapshot[applied={T|T < 7334621901587566592}]
I20260929 10:52:55.196017 4317 mvcc.cc:205] Tried to move back new op lower bound from 7334621901587566592 to 7334621901518286848. Current Snapshot: MvccSnapshot[applied={T|T < 7334621901587566592}]
I20260929 10:52:55.196278 4316 mvcc.cc:205] Tried to move back new op lower bound from 7334621901587566592 to 7334621901518286848. Current Snapshot: MvccSnapshot[applied={T|T < 7334621901587566592}]
I20260929 10:52:55.198032 3821 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:52:55.198133 3821 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:52:55.198921 3821 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537: Sending DeleteTablet for 3 replicas of tablet 79b47648b6f046438dafa78af273da05
I20260929 10:52:55.199250 4218 tablet_service.cc:1514] Processing DeleteTablet for tablet 79b47648b6f046438dafa78af273da05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:52:55 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:36364
I20260929 10:52:55.199318 3949 tablet_service.cc:1514] Processing DeleteTablet for tablet 79b47648b6f046438dafa78af273da05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:52:55 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:41722
I20260929 10:52:55.199368 4084 tablet_service.cc:1514] Processing DeleteTablet for tablet 79b47648b6f046438dafa78af273da05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:52:55 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:38882
I20260929 10:52:55.199581 4328 tablet_replica.cc:330] T 79b47648b6f046438dafa78af273da05 P 8ba6deb8d47147268bd65549332255ba: stopping tablet replica
I20260929 10:52:55.199584 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 3885
I20260929 10:52:55.200491 4328 raft_consensus.cc:2229] T 79b47648b6f046438dafa78af273da05 P 8ba6deb8d47147268bd65549332255ba [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:52:55.200621 4328 raft_consensus.cc:2258] T 79b47648b6f046438dafa78af273da05 P 8ba6deb8d47147268bd65549332255ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:52:55.201112 4328 ts_tablet_manager.cc:1905] T 79b47648b6f046438dafa78af273da05 P 8ba6deb8d47147268bd65549332255ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:52:55.201880 4325 tablet_replica.cc:330] T 79b47648b6f046438dafa78af273da05 P ee54fe9fec6d4197862fa22f8a07fa35: stopping tablet replica
I20260929 10:52:55.202004 4325 raft_consensus.cc:2229] T 79b47648b6f046438dafa78af273da05 P ee54fe9fec6d4197862fa22f8a07fa35 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:52:55.202142 4325 raft_consensus.cc:2258] T 79b47648b6f046438dafa78af273da05 P ee54fe9fec6d4197862fa22f8a07fa35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:52:55.202605 4325 ts_tablet_manager.cc:1905] T 79b47648b6f046438dafa78af273da05 P ee54fe9fec6d4197862fa22f8a07fa35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:52:55.202981 4328 ts_tablet_manager.cc:1918] T 79b47648b6f046438dafa78af273da05 P 8ba6deb8d47147268bd65549332255ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:52:55.203058 4328 log.cc:1196] T 79b47648b6f046438dafa78af273da05 P 8ba6deb8d47147268bd65549332255ba: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/wal/wals/79b47648b6f046438dafa78af273da05
I20260929 10:52:55.203339 4328 ts_tablet_manager.cc:1939] T 79b47648b6f046438dafa78af273da05 P 8ba6deb8d47147268bd65549332255ba: Deleting consensus metadata
I20260929 10:52:55.203995 3810 catalog_manager.cc:4841] TS 8ba6deb8d47147268bd65549332255ba (127.20.80.194:39931): tablet 79b47648b6f046438dafa78af273da05 (table test-table [id=5599160681454b819ce5a812324148fc]) successfully deleted
I20260929 10:52:55.204607 4325 ts_tablet_manager.cc:1918] T 79b47648b6f046438dafa78af273da05 P ee54fe9fec6d4197862fa22f8a07fa35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:52:55.204708 4325 log.cc:1196] T 79b47648b6f046438dafa78af273da05 P ee54fe9fec6d4197862fa22f8a07fa35: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/wal/wals/79b47648b6f046438dafa78af273da05
I20260929 10:52:55.204969 4325 ts_tablet_manager.cc:1939] T 79b47648b6f046438dafa78af273da05 P ee54fe9fec6d4197862fa22f8a07fa35: Deleting consensus metadata
I20260929 10:52:55.205400 3809 catalog_manager.cc:4841] TS ee54fe9fec6d4197862fa22f8a07fa35 (127.20.80.195:39941): tablet 79b47648b6f046438dafa78af273da05 (table test-table [id=5599160681454b819ce5a812324148fc]) successfully deleted
W20260929 10:52:55.211830 3807 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.80.193:44365 (error 108)
I20260929 10:52:55.211989 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 4020
I20260929 10:52:55.213688 4297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:52:55.213565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:44365 (local address 127.0.0.1:41734)
0929 10:52:55.213608 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0929 10:52:55.213647 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.193:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260929 10:52:55.213836 3807 catalog_manager.cc:4584] TS 6068e15b32f344179f1972d1fe7e3c52 (127.20.80.193:44365): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 79b47648b6f046438dafa78af273da05: Network error: Client connection negotiation failed: client connection to 127.20.80.193:44365: connect: Connection refused (error 111)
I20260929 10:52:55.217257 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 4155
I20260929 10:52:55.221932 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 3791
I20260929 10:53:15.227418 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:39195
--webserver_interface=127.20.80.254
--webserver_port=42113
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:46069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:39195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:53:15.312546 4330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:53:15.312763 4330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:53:15.312788 4330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:53:15.313907 4330 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 10:53:15.313949 4330 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 10:53:15.313964 4330 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 10:53:15.313978 4330 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:53:15.313993 4330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:53:15.314008 4330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:53:15.314029 4330 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:53:15.314044 4330 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:53:15.315474 4330 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:46069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:39195
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:39195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=42113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:53:15.315666 4330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:53:15.315838 4330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:53:15.325111 4339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:53:15.325198 4330 server_base.cc:1034] running on GCE node
W20260929 10:53:15.325155 4336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:15.325111 4337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:53:15.325717 4330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:53:15.326047 4330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:53:15.327229 4330 hybrid_clock.cc:648] HybridClock initialized: now 1790679195327213 us; error 30 us; skew 500 ppm
Sep 29 10:53:15 dist-test-slave-vqch krb5kdc[3778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679195, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:53:15.335327 4330 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:53:15.335805 4330 webserver.cc:460] Webserver started at http://127.20.80.254:42113/ using document root <none> and password file <none>
I20260929 10:53:15.336050 4330 fs_manager.cc:359] Metadata directory not provided
I20260929 10:53:15.336136 4330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:53:15.337637 4330 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:53:15.338413 4346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:15.338622 4330 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:53:15.338783 4330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "a1657adc941042819338bbf51f53f537"
format_stamp: "Formatted at 2026-09-29 10:52:54 on dist-test-slave-vqch"
server_key: "baeaf904359347022e9a2a8d6e8ec9ad"
server_key_iv: "1577570795ac1af689bf3c04ebc2732e"
server_key_version: "encryptionkey@0"
I20260929 10:53:15.339210 4330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:53:15.346423 4330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:53:15.348393 4330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:53:15.348551 4330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:53:15.352742 4330 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:39195
I20260929 10:53:15.352793 4398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:39195 every 8 connection(s)
I20260929 10:53:15.353217 4330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:53:15.354760 4399 sys_catalog.cc:263] Verifying existing consensus state
I20260929 10:53:15.355415 4399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537: Bootstrap starting.
I20260929 10:53:15.362866 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 4330
I20260929 10:53:15.363168 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.193:44365
--local_ip_for_outbound_sockets=127.20.80.193
--tserver_master_addrs=127.20.80.254:39195
--webserver_port=36339
--webserver_interface=127.20.80.193
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:46069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:53:15.364231 4399 log.cc:824] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537: Log is configured to *not* fsync() on all Append() calls
I20260929 10:53:15.366153 4399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 10:53:15.366501 4399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537: Bootstrap complete.
I20260929 10:53:15.368742 4399 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1657adc941042819338bbf51f53f537" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 39195 } }
I20260929 10:53:15.368986 4399 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1657adc941042819338bbf51f53f537, State: Initialized, Role: FOLLOWER
I20260929 10:53:15.369189 4399 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1657adc941042819338bbf51f53f537" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 39195 } }
I20260929 10:53:15.369268 4399 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:53:15.369324 4399 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:53:15.369370 4399 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [term 1 FOLLOWER]: Advancing to term 2
I20260929 10:53:15.370358 4399 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1657adc941042819338bbf51f53f537" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 39195 } }
I20260929 10:53:15.370479 4399 leader_election.cc:302] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1657adc941042819338bbf51f53f537; no voters:
I20260929 10:53:15.370675 4399 leader_election.cc:288] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 10:53:15.370711 4404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 10:53:15.370829 4404 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [term 2 LEADER]: Becoming Leader. State: Replica: a1657adc941042819338bbf51f53f537, State: Running, Role: LEADER
I20260929 10:53:15.370950 4399 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:53:15.370975 4404 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1657adc941042819338bbf51f53f537" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 39195 } }
I20260929 10:53:15.371259 4404 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1657adc941042819338bbf51f53f537. Latest consensus state: current_term: 2 leader_uuid: "a1657adc941042819338bbf51f53f537" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1657adc941042819338bbf51f53f537" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 39195 } } }
I20260929 10:53:15.371266 4405 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a1657adc941042819338bbf51f53f537" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1657adc941042819338bbf51f53f537" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 39195 } } }
I20260929 10:53:15.371507 4405 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [sys.catalog]: This master's current role is: LEADER
I20260929 10:53:15.371732 4413 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:53:15.371429 4404 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [sys.catalog]: This master's current role is: LEADER
I20260929 10:53:15.372675 4413 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:53:15.372823 4413 catalog_manager.cc:1259] Loaded cluster ID: e51b1a7e7f214bc48fb8541e0ad3972e
I20260929 10:53:15.372864 4413 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:53:15.374086 4413 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:53:15.374277 4413 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537: Loaded TSK: 0
I20260929 10:53:15.386101 4413 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537: Generated new TSK 1
I20260929 10:53:15.386212 4413 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 10:53:15.448318 4402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:53:15.448551 4402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:53:15.448592 4402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:53:15.449872 4402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:53:15.449936 4402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:53:15.449981 4402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.193
I20260929 10:53:15.451459 4402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:46069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.193:44365
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.80.193
--webserver_port=36339
--tserver_master_addrs=127.20.80.254:39195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.193
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:53:15.451725 4402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:53:15.451968 4402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:53:15.458042 4428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:15.458060 4427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:15.458058 4430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:53:15.458294 4402 server_base.cc:1034] running on GCE node
I20260929 10:53:15.458616 4402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:53:15.458877 4402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:53:15.460031 4402 hybrid_clock.cc:648] HybridClock initialized: now 1790679195460020 us; error 27 us; skew 500 ppm
Sep 29 10:53:15 dist-test-slave-vqch krb5kdc[3778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679195, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:53:15.462749 4402 init.cc:377] Logged in from keytab as kudu/127.20.80.193@KRBTEST.COM (short username kudu)
I20260929 10:53:15.463268 4402 webserver.cc:460] Webserver started at http://127.20.80.193:36339/ using document root <none> and password file <none>
I20260929 10:53:15.463572 4402 fs_manager.cc:359] Metadata directory not provided
I20260929 10:53:15.463654 4402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:53:15.465377 4402 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:53:15.466252 4437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:15.466475 4402 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:53:15.466568 4402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/wal
uuid: "6068e15b32f344179f1972d1fe7e3c52"
format_stamp: "Formatted at 2026-09-29 10:52:54 on dist-test-slave-vqch"
server_key: "a243feabe40acd714bdb8f24f725bb07"
server_key_iv: "e19c45bb6b848a94639cac5062fab8ae"
server_key_version: "encryptionkey@0"
I20260929 10:53:15.467096 4402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:53:15.482753 4402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:53:15.485314 4402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:53:15.485546 4402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:53:15.485852 4402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:53:15.486370 4444 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 10:53:15.487831 4402 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 10:53:15.487896 4402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260929 10:53:15.487934 4402 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 10:53:15.488860 4402 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 10:53:15.488910 4402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260929 10:53:15.488974 4444 tablet_bootstrap.cc:492] T 79b47648b6f046438dafa78af273da05 P 6068e15b32f344179f1972d1fe7e3c52: Bootstrap starting.
I20260929 10:53:15.496897 4402 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.193:44365
I20260929 10:53:15.497402 4402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
I20260929 10:53:15.498250 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 4402
I20260929 10:53:15.498572 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.194:39931
--local_ip_for_outbound_sockets=127.20.80.194
--tserver_master_addrs=127.20.80.254:39195
--webserver_port=45783
--webserver_interface=127.20.80.194
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:46069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:53:15.500855 4444 log.cc:824] T 79b47648b6f046438dafa78af273da05 P 6068e15b32f344179f1972d1fe7e3c52: Log is configured to *not* fsync() on all Append() calls
I20260929 10:53:15.502611 4552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.193:44365 every 8 connection(s)
I20260929 10:53:15.503579 4444 tablet_bootstrap.cc:492] T 79b47648b6f046438dafa78af273da05 P 6068e15b32f344179f1972d1fe7e3c52: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 10:53:15.503906 4444 tablet_bootstrap.cc:492] T 79b47648b6f046438dafa78af273da05 P 6068e15b32f344179f1972d1fe7e3c52: Bootstrap complete.
I20260929 10:53:15.505126 4444 ts_tablet_manager.cc:1397] T 79b47648b6f046438dafa78af273da05 P 6068e15b32f344179f1972d1fe7e3c52: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.010s
I20260929 10:53:15.506589 4444 raft_consensus.cc:348] T 79b47648b6f046438dafa78af273da05 P 6068e15b32f344179f1972d1fe7e3c52 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6068e15b32f344179f1972d1fe7e3c52" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 44365 } } peers { permanent_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39941 } } peers { permanent_uuid: "8ba6deb8d47147268bd65549332255ba" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 39931 } }
I20260929 10:53:15.506929 4444 raft_consensus.cc:729] T 79b47648b6f046438dafa78af273da05 P 6068e15b32f344179f1972d1fe7e3c52 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6068e15b32f344179f1972d1fe7e3c52, State: Initialized, Role: FOLLOWER
I20260929 10:53:15.507064 4444 consensus_queue.cc:260] T 79b47648b6f046438dafa78af273da05 P 6068e15b32f344179f1972d1fe7e3c52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6068e15b32f344179f1972d1fe7e3c52" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 44365 } } peers { permanent_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39941 } } peers { permanent_uuid: "8ba6deb8d47147268bd65549332255ba" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 39931 } }
I20260929 10:53:15.507337 4444 ts_tablet_manager.cc:1428] T 79b47648b6f046438dafa78af273da05 P 6068e15b32f344179f1972d1fe7e3c52: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
Sep 29 10:53:15 dist-test-slave-vqch krb5kdc[3778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679195, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:53:15.515085 4555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:15.498143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.193:56777)
0929 10:53:15.501159 (+ 3016us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:15.501210 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:15.501732 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:15.502449 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:15.502453 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:15.502848 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:53:15.503067 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:15.503072 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.504957 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.504959 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:15.506512 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:15.506516 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.507710 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.507712 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:15.507855 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:15.508637 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:53:15.508647 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:53:15.510617 (+ 1970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:53:15.513216 (+ 2599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:53:15.513224 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:53:15.513226 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:53:15.513524 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:53:15.513847 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:53:15.513848 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:53:15.513848 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:53:15.513973 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:53:15.514457 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:53:15.514462 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:53:15.514613 (+ 151us) client_negotiation.cc:769] Sending connection context
0929 10:53:15.514711 (+ 98us) client_negotiation.cc:240] Negotiation successful
0929 10:53:15.514879 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2966,"thread_start_us":103,"threads_started":1}
I20260929 10:53:15.515429 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:15.498426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:56777 (local address 127.20.80.254:39195)
0929 10:53:15.502055 (+ 3629us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:15.502064 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:15.502095 (+ 31us) server_negotiation.cc:407] Connection header received
0929 10:53:15.502204 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:15.502209 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:15.502250 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:15.502404 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:53:15.503187 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.504823 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.506618 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.507604 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.507754 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:15.510766 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:53:15.510771 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:53:15.510771 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:53:15.510784 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:53:15.513103 (+ 2319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:15.513643 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:15.513646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:15.513647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:15.513772 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:15.514056 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:15.514057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:15.514058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:15.514372 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:53:15.514471 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:15.514801 (+ 330us) server_negotiation.cc:299] Negotiation successful
0929 10:53:15.514942 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3580,"thread_start_us":87,"threads_started":1}
I20260929 10:53:15.516101 4553 heartbeater.cc:347] Connected to a master server at 127.20.80.254:39195
I20260929 10:53:15.516201 4553 heartbeater.cc:464] Registering TS with master...
I20260929 10:53:15.516372 4553 heartbeater.cc:511] Master 127.20.80.254:39195 requested a full tablet report, sending...
I20260929 10:53:15.517199 4363 ts_manager.cc:194] Registered new tserver with Master: 6068e15b32f344179f1972d1fe7e3c52 (127.20.80.193:44365)
I20260929 10:53:15.518411 4363 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.193@KRBTEST.COM'} at 127.20.80.193:56777
I20260929 10:53:15.519420 4553 heartbeater.cc:503] Master 127.20.80.254:39195 was elected leader, sending a full tablet report...
I20260929 10:53:15.524890 4560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:15.517817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:44365 (local address 127.0.0.1:35258)
0929 10:53:15.518112 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:15.518151 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:15.518225 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:15.521282 (+ 3057us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:15.521286 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:15.521289 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:53:15.521359 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:15.521361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.522602 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.522603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:15.523771 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:15.523774 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.524640 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.524642 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:15.524712 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:15.524723 (+ 11us) client_negotiation.cc:769] Sending connection context
0929 10:53:15.524792 (+ 69us) client_negotiation.cc:240] Negotiation successful
0929 10:53:15.524833 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":96,"threads_started":1}
I20260929 10:53:15.524927 4561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:15.518067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35258 (local address 127.20.80.193:44365)
0929 10:53:15.521089 (+ 3022us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:15.521094 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:15.521105 (+ 11us) server_negotiation.cc:407] Connection header received
0929 10:53:15.521139 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:15.521141 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:15.521171 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:15.521254 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:53:15.521442 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.522510 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.523856 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.524538 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.524612 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:15.524665 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:15.524818 (+ 153us) server_negotiation.cc:299] Negotiation successful
0929 10:53:15.524859 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2989,"thread_start_us":73,"threads_started":1}
I20260929 10:53:15.525426 4487 tablet_service.cc:1514] Processing DeleteTablet for tablet 79b47648b6f046438dafa78af273da05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:52:55 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:35258
I20260929 10:53:15.525987 4562 tablet_replica.cc:330] stopping tablet replica
I20260929 10:53:15.526090 4562 raft_consensus.cc:2229] T 79b47648b6f046438dafa78af273da05 P 6068e15b32f344179f1972d1fe7e3c52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:53:15.526160 4562 raft_consensus.cc:2258] T 79b47648b6f046438dafa78af273da05 P 6068e15b32f344179f1972d1fe7e3c52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:53:15.526469 4562 ts_tablet_manager.cc:1905] T 79b47648b6f046438dafa78af273da05 P 6068e15b32f344179f1972d1fe7e3c52: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:53:15.528694 4562 ts_tablet_manager.cc:1918] T 79b47648b6f046438dafa78af273da05 P 6068e15b32f344179f1972d1fe7e3c52: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:53:15.528757 4562 log.cc:1196] T 79b47648b6f046438dafa78af273da05 P 6068e15b32f344179f1972d1fe7e3c52: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/wal/wals/79b47648b6f046438dafa78af273da05
I20260929 10:53:15.529023 4562 ts_tablet_manager.cc:1939] T 79b47648b6f046438dafa78af273da05 P 6068e15b32f344179f1972d1fe7e3c52: Deleting consensus metadata
I20260929 10:53:15.529508 4347 catalog_manager.cc:4841] TS 6068e15b32f344179f1972d1fe7e3c52 (127.20.80.193:44365): tablet 79b47648b6f046438dafa78af273da05 (table test-table [id=5599160681454b819ce5a812324148fc]) successfully deleted
W20260929 10:53:15.587430 4557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:53:15.587661 4557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:53:15.587698 4557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:53:15.589006 4557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:53:15.589061 4557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:53:15.589104 4557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.194
I20260929 10:53:15.590498 4557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:46069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.194:39931
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.80.194
--webserver_port=45783
--tserver_master_addrs=127.20.80.254:39195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.194
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:53:15.590749 4557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:53:15.590978 4557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:53:15.596036 4570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:15.596045 4568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:53:15.596218 4557 server_base.cc:1034] running on GCE node
W20260929 10:53:15.596256 4567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:53:15.596479 4557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:53:15.596717 4557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:53:15.597904 4557 hybrid_clock.cc:648] HybridClock initialized: now 1790679195597889 us; error 29 us; skew 500 ppm
Sep 29 10:53:15 dist-test-slave-vqch krb5kdc[3778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679195, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:53:15.600410 4557 init.cc:377] Logged in from keytab as kudu/127.20.80.194@KRBTEST.COM (short username kudu)
I20260929 10:53:15.600838 4557 webserver.cc:460] Webserver started at http://127.20.80.194:45783/ using document root <none> and password file <none>
I20260929 10:53:15.601109 4557 fs_manager.cc:359] Metadata directory not provided
I20260929 10:53:15.601186 4557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:53:15.602524 4557 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:53:15.603163 4577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:15.603346 4557 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:53:15.603442 4557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/wal
uuid: "8ba6deb8d47147268bd65549332255ba"
format_stamp: "Formatted at 2026-09-29 10:52:54 on dist-test-slave-vqch"
server_key: "b099fc61df5a00b45a75c1142b1fb82f"
server_key_iv: "748e729b9178b339e4e0e5ef01d61646"
server_key_version: "encryptionkey@0"
I20260929 10:53:15.603801 4557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:53:15.618688 4557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:53:15.620563 4557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:53:15.620720 4557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:53:15.620946 4557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:53:15.621304 4557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:53:15.621356 4557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:15.621395 4557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:53:15.621425 4557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:15.627316 4557 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.194:39931
I20260929 10:53:15.627350 4691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.194:39931 every 8 connection(s)
I20260929 10:53:15.627693 4557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
Sep 29 10:53:15 dist-test-slave-vqch krb5kdc[3778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679195, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:53:15.636677 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 4557
I20260929 10:53:15.637013 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.195:39941
--local_ip_for_outbound_sockets=127.20.80.195
--tserver_master_addrs=127.20.80.254:39195
--webserver_port=39871
--webserver_interface=127.20.80.195
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:46069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:53:15.637042 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:15.628408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:46625 (local address 127.20.80.254:39195)
0929 10:53:15.628489 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:15.628492 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:15.628718 (+ 226us) server_negotiation.cc:407] Connection header received
0929 10:53:15.629209 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:15.629210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:15.629221 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:15.629275 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:53:15.630204 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.630706 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.631687 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.631902 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.631961 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:15.634155 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:53:15.634156 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:53:15.634157 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:53:15.634163 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:53:15.635579 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:15.635958 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:15.635959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:15.635960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:15.636019 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:15.636246 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:15.636246 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:15.636247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:15.636371 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:53:15.636428 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:15.636830 (+ 402us) server_negotiation.cc:299] Negotiation successful
0929 10:53:15.636924 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 10:53:15.637331 4694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:15.628401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.194:46625)
0929 10:53:15.628594 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:15.628629 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:15.629123 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:15.629342 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:15.629345 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:15.629905 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:53:15.630095 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:15.630100 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.630799 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.630800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:15.631596 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:15.631598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.631976 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.631976 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:15.632098 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:15.632680 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:53:15.632688 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:53:15.634069 (+ 1381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:53:15.635646 (+ 1577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:53:15.635652 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:53:15.635654 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:53:15.635878 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:53:15.636090 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:53:15.636090 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:53:15.636091 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:53:15.636185 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:53:15.636427 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:53:15.636432 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:53:15.636620 (+ 188us) client_negotiation.cc:769] Sending connection context
0929 10:53:15.636746 (+ 126us) client_negotiation.cc:240] Negotiation successful
0929 10:53:15.636968 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":114,"threads_started":1}
I20260929 10:53:15.637856 4692 heartbeater.cc:347] Connected to a master server at 127.20.80.254:39195
I20260929 10:53:15.637957 4692 heartbeater.cc:464] Registering TS with master...
I20260929 10:53:15.638154 4692 heartbeater.cc:511] Master 127.20.80.254:39195 requested a full tablet report, sending...
I20260929 10:53:15.638505 4363 ts_manager.cc:194] Registered new tserver with Master: 8ba6deb8d47147268bd65549332255ba (127.20.80.194:39931)
I20260929 10:53:15.638892 4363 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.194@KRBTEST.COM'} at 127.20.80.194:46625
W20260929 10:53:15.718884 4695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:53:15.719113 4695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:53:15.719152 4695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:53:15.720287 4695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:53:15.720348 4695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:53:15.720392 4695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.195
I20260929 10:53:15.722347 4695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:46069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.195:39941
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.80.195
--webserver_port=39871
--tserver_master_addrs=127.20.80.254:39195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.195
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:53:15.722646 4695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:53:15.722884 4695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 10:53:15.728036 4695 server_base.cc:1034] running on GCE node
W20260929 10:53:15.728019 4703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:15.728039 4701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:15.728021 4700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:53:15.728441 4695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:53:15.728675 4695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:53:15.729884 4695 hybrid_clock.cc:648] HybridClock initialized: now 1790679195729863 us; error 31 us; skew 500 ppm
Sep 29 10:53:15 dist-test-slave-vqch krb5kdc[3778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679195, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:53:15.732415 4695 init.cc:377] Logged in from keytab as kudu/127.20.80.195@KRBTEST.COM (short username kudu)
I20260929 10:53:15.732856 4695 webserver.cc:460] Webserver started at http://127.20.80.195:39871/ using document root <none> and password file <none>
I20260929 10:53:15.733089 4695 fs_manager.cc:359] Metadata directory not provided
I20260929 10:53:15.733148 4695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:53:15.734407 4695 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:53:15.734929 4710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:15.735080 4695 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:53:15.735162 4695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/wal
uuid: "ee54fe9fec6d4197862fa22f8a07fa35"
format_stamp: "Formatted at 2026-09-29 10:52:55 on dist-test-slave-vqch"
server_key: "9c6042a48842f0a37e6165c171fd5bf8"
server_key_iv: "93102fa2138c42adf8f2c705798eb20b"
server_key_version: "encryptionkey@0"
I20260929 10:53:15.735481 4695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:53:15.751554 4695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:53:15.753724 4695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:53:15.753846 4695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:53:15.754053 4695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:53:15.754317 4695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:53:15.754345 4695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:15.754369 4695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:53:15.754380 4695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:15.759764 4695 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.195:39941
I20260929 10:53:15.759989 4824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.195:39941 every 8 connection(s)
I20260929 10:53:15.760181 4695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
I20260929 10:53:15.762291 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 4695
Sep 29 10:53:15 dist-test-slave-vqch krb5kdc[3778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679195, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:53:15.769886 4828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:15.762714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59606 (local address 127.20.80.254:39195)
0929 10:53:15.762844 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:15.762846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:15.762961 (+ 115us) server_negotiation.cc:407] Connection header received
0929 10:53:15.763028 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:15.763029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:15.763041 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:15.763100 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:53:15.763813 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.764274 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.765285 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.765577 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.765640 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:15.766855 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:53:15.766857 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:53:15.766857 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:53:15.766865 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:53:15.768602 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:15.769070 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:15.769071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:15.769072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:15.769130 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:15.769353 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:15.769354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:15.769354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:15.769466 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:53:15.769651 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:15.769689 (+ 38us) server_negotiation.cc:299] Negotiation successful
0929 10:53:15.769773 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":68,"threads_started":1}
I20260929 10:53:15.770409 4827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:15.760888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.195:45269)
0929 10:53:15.761070 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:15.761115 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:15.761615 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:15.761818 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:15.761822 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:15.762455 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:53:15.762688 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:15.762696 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.763434 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.763435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:15.764475 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:15.764478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.764856 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.764857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:15.765014 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:15.765625 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:53:15.765636 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:53:15.767267 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:53:15.768896 (+ 1629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:53:15.768901 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:53:15.768903 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:53:15.769183 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:53:15.769397 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:53:15.769397 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:53:15.769398 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:53:15.769510 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:53:15.769751 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:53:15.769758 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:53:15.769940 (+ 182us) client_negotiation.cc:769] Sending connection context
0929 10:53:15.770037 (+ 97us) client_negotiation.cc:240] Negotiation successful
0929 10:53:15.770219 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":116,"threads_started":1}
I20260929 10:53:15.770780 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:15.760888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:45269 (local address 127.20.80.254:39195)
0929 10:53:15.760948 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:15.760950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:15.761203 (+ 253us) server_negotiation.cc:407] Connection header received
0929 10:53:15.761699 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:15.761700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:15.761711 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:15.761755 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:53:15.762803 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.763336 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.764563 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.764750 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.764839 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:15.767356 (+ 2517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:53:15.767357 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:53:15.767357 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:53:15.767364 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:53:15.768815 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:15.769271 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:15.769272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:15.769273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:15.769334 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:15.769581 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:15.769582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:15.769582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:15.769688 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:53:15.769758 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:15.770573 (+ 815us) server_negotiation.cc:299] Negotiation successful
0929 10:53:15.770659 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 10:53:15.771071 4825 heartbeater.cc:347] Connected to a master server at 127.20.80.254:39195
I20260929 10:53:15.771198 4825 heartbeater.cc:464] Registering TS with master...
I20260929 10:53:15.771368 4825 heartbeater.cc:511] Master 127.20.80.254:39195 requested a full tablet report, sending...
I20260929 10:53:15.771716 4363 ts_manager.cc:194] Registered new tserver with Master: ee54fe9fec6d4197862fa22f8a07fa35 (127.20.80.195:39941)
I20260929 10:53:15.772073 4363 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.195@KRBTEST.COM'} at 127.20.80.195:45269
I20260929 10:53:15.781733 20803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:53:15.784464 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:15.782086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59610 (local address 127.20.80.254:39195)
0929 10:53:15.782144 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:15.782146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:15.782235 (+ 89us) server_negotiation.cc:407] Connection header received
0929 10:53:15.782313 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:15.782314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:15.782349 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:15.782407 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:53:15.782507 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.782982 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.783747 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.783930 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.783982 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:15.784109 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:53:15.784204 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:53:15.784265 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:15.784368 (+ 103us) server_negotiation.cc:299] Negotiation successful
0929 10:53:15.784383 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 10:53:15.784794 4363 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:59610:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:53:15.786458 4363 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:53:15.788503 4487 tablet_service.cc:1467] Processing CreateTablet for tablet 09a5c9c3f7d94b5bad74b87caf975912 (DEFAULT_TABLE table=test-table [id=d5b4025e0aad4de7836d8ad752d6edb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:53:15.788709 4487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09a5c9c3f7d94b5bad74b87caf975912. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:53:15.790104 4444 tablet_bootstrap.cc:492] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52: Bootstrap starting.
I20260929 10:53:15.790488 4444 tablet_bootstrap.cc:654] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52: Neither blocks nor log segments found. Creating new log.
I20260929 10:53:15.791075 4444 tablet_bootstrap.cc:492] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52: No bootstrap required, opened a new log
I20260929 10:53:15.791131 4444 ts_tablet_manager.cc:1397] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260929 10:53:15.791242 4444 raft_consensus.cc:348] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba6deb8d47147268bd65549332255ba" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 39931 } } peers { permanent_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39941 } } peers { permanent_uuid: "6068e15b32f344179f1972d1fe7e3c52" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 44365 } }
I20260929 10:53:15.791316 4444 raft_consensus.cc:374] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:53:15.791373 4444 raft_consensus.cc:729] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6068e15b32f344179f1972d1fe7e3c52, State: Initialized, Role: FOLLOWER
I20260929 10:53:15.791417 4444 consensus_queue.cc:260] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba6deb8d47147268bd65549332255ba" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 39931 } } peers { permanent_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39941 } } peers { permanent_uuid: "6068e15b32f344179f1972d1fe7e3c52" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 44365 } }
I20260929 10:53:15.791495 4444 ts_tablet_manager.cc:1428] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:15.792423 4560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:15.788214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:39931 (local address 127.0.0.1:44752)
0929 10:53:15.788254 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:15.788265 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:15.788300 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:15.788682 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:15.788684 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:15.788686 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:53:15.788728 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:15.788729 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.790004 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.790005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:15.791079 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:15.791082 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.792167 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.792169 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:15.792230 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:15.792234 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:53:15.792292 (+ 58us) client_negotiation.cc:240] Negotiation successful
0929 10:53:15.792338 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260929 10:53:15.792692 4831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:15.788309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44752 (local address 127.20.80.194:39931)
0929 10:53:15.788487 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:15.788492 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:15.788504 (+ 12us) server_negotiation.cc:407] Connection header received
0929 10:53:15.788548 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:15.788551 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:15.788567 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:15.788807 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:53:15.788906 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.789763 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.791195 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.791986 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.792063 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:15.792121 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:15.792546 (+ 425us) server_negotiation.cc:299] Negotiation successful
0929 10:53:15.792593 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":104,"threads_started":1}
I20260929 10:53:15.793395 4626 tablet_service.cc:1467] Processing CreateTablet for tablet 09a5c9c3f7d94b5bad74b87caf975912 (DEFAULT_TABLE table=test-table [id=d5b4025e0aad4de7836d8ad752d6edb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:53:15.793758 4626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09a5c9c3f7d94b5bad74b87caf975912. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:53:15.794258 4833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:15.788222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:39941 (local address 127.0.0.1:42506)
0929 10:53:15.790222 (+ 2000us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:15.790232 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:15.790283 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:15.790570 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:15.790571 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:15.790573 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:53:15.790613 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:15.790615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.791886 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.791887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:15.793176 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:15.793179 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.794066 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.794067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:15.794116 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:15.794120 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:53:15.794172 (+ 52us) client_negotiation.cc:240] Negotiation successful
0929 10:53:15.794203 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1978,"mutex_wait_us":499,"thread_start_us":87,"threads_started":1}
I20260929 10:53:15.794330 4832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:15.788517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42506 (local address 127.20.80.195:39941)
0929 10:53:15.788906 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:15.788911 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:15.790345 (+ 1434us) server_negotiation.cc:407] Connection header received
0929 10:53:15.790391 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:15.790394 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:15.790411 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:15.790483 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:53:15.790672 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.791815 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.793256 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.793946 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.794023 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:15.794074 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:15.794200 (+ 126us) server_negotiation.cc:299] Negotiation successful
0929 10:53:15.794239 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":106,"threads_started":1}
I20260929 10:53:15.794953 4758 tablet_service.cc:1467] Processing CreateTablet for tablet 09a5c9c3f7d94b5bad74b87caf975912 (DEFAULT_TABLE table=test-table [id=d5b4025e0aad4de7836d8ad752d6edb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:53:15.795240 4758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09a5c9c3f7d94b5bad74b87caf975912. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:53:15.796700 4835 tablet_bootstrap.cc:492] T 09a5c9c3f7d94b5bad74b87caf975912 P 8ba6deb8d47147268bd65549332255ba: Bootstrap starting.
I20260929 10:53:15.797349 4835 tablet_bootstrap.cc:654] T 09a5c9c3f7d94b5bad74b87caf975912 P 8ba6deb8d47147268bd65549332255ba: Neither blocks nor log segments found. Creating new log.
I20260929 10:53:15.797885 4835 log.cc:824] T 09a5c9c3f7d94b5bad74b87caf975912 P 8ba6deb8d47147268bd65549332255ba: Log is configured to *not* fsync() on all Append() calls
I20260929 10:53:15.797906 4836 tablet_bootstrap.cc:492] T 09a5c9c3f7d94b5bad74b87caf975912 P ee54fe9fec6d4197862fa22f8a07fa35: Bootstrap starting.
I20260929 10:53:15.798530 4836 tablet_bootstrap.cc:654] T 09a5c9c3f7d94b5bad74b87caf975912 P ee54fe9fec6d4197862fa22f8a07fa35: Neither blocks nor log segments found. Creating new log.
I20260929 10:53:15.798660 4835 tablet_bootstrap.cc:492] T 09a5c9c3f7d94b5bad74b87caf975912 P 8ba6deb8d47147268bd65549332255ba: No bootstrap required, opened a new log
I20260929 10:53:15.798738 4835 ts_tablet_manager.cc:1397] T 09a5c9c3f7d94b5bad74b87caf975912 P 8ba6deb8d47147268bd65549332255ba: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:53:15.799119 4836 log.cc:824] T 09a5c9c3f7d94b5bad74b87caf975912 P ee54fe9fec6d4197862fa22f8a07fa35: Log is configured to *not* fsync() on all Append() calls
I20260929 10:53:15.799860 4836 tablet_bootstrap.cc:492] T 09a5c9c3f7d94b5bad74b87caf975912 P ee54fe9fec6d4197862fa22f8a07fa35: No bootstrap required, opened a new log
I20260929 10:53:15.799924 4836 ts_tablet_manager.cc:1397] T 09a5c9c3f7d94b5bad74b87caf975912 P ee54fe9fec6d4197862fa22f8a07fa35: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:53:15.800753 4835 raft_consensus.cc:348] T 09a5c9c3f7d94b5bad74b87caf975912 P 8ba6deb8d47147268bd65549332255ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba6deb8d47147268bd65549332255ba" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 39931 } } peers { permanent_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39941 } } peers { permanent_uuid: "6068e15b32f344179f1972d1fe7e3c52" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 44365 } }
I20260929 10:53:15.800945 4835 raft_consensus.cc:374] T 09a5c9c3f7d94b5bad74b87caf975912 P 8ba6deb8d47147268bd65549332255ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:53:15.801010 4835 raft_consensus.cc:729] T 09a5c9c3f7d94b5bad74b87caf975912 P 8ba6deb8d47147268bd65549332255ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ba6deb8d47147268bd65549332255ba, State: Initialized, Role: FOLLOWER
I20260929 10:53:15.801132 4835 consensus_queue.cc:260] T 09a5c9c3f7d94b5bad74b87caf975912 P 8ba6deb8d47147268bd65549332255ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba6deb8d47147268bd65549332255ba" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 39931 } } peers { permanent_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39941 } } peers { permanent_uuid: "6068e15b32f344179f1972d1fe7e3c52" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 44365 } }
I20260929 10:53:15.801422 4835 ts_tablet_manager.cc:1428] T 09a5c9c3f7d94b5bad74b87caf975912 P 8ba6deb8d47147268bd65549332255ba: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 10:53:15.801453 4692 heartbeater.cc:503] Master 127.20.80.254:39195 was elected leader, sending a full tablet report...
I20260929 10:53:15.801827 4836 raft_consensus.cc:348] T 09a5c9c3f7d94b5bad74b87caf975912 P ee54fe9fec6d4197862fa22f8a07fa35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba6deb8d47147268bd65549332255ba" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 39931 } } peers { permanent_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39941 } } peers { permanent_uuid: "6068e15b32f344179f1972d1fe7e3c52" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 44365 } }
I20260929 10:53:15.801973 4836 raft_consensus.cc:374] T 09a5c9c3f7d94b5bad74b87caf975912 P ee54fe9fec6d4197862fa22f8a07fa35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:53:15.802016 4836 raft_consensus.cc:729] T 09a5c9c3f7d94b5bad74b87caf975912 P ee54fe9fec6d4197862fa22f8a07fa35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee54fe9fec6d4197862fa22f8a07fa35, State: Initialized, Role: FOLLOWER
I20260929 10:53:15.802165 4836 consensus_queue.cc:260] T 09a5c9c3f7d94b5bad74b87caf975912 P ee54fe9fec6d4197862fa22f8a07fa35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba6deb8d47147268bd65549332255ba" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 39931 } } peers { permanent_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39941 } } peers { permanent_uuid: "6068e15b32f344179f1972d1fe7e3c52" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 44365 } }
I20260929 10:53:15.802439 4825 heartbeater.cc:503] Master 127.20.80.254:39195 was elected leader, sending a full tablet report...
I20260929 10:53:15.802459 4836 ts_tablet_manager.cc:1428] T 09a5c9c3f7d94b5bad74b87caf975912 P ee54fe9fec6d4197862fa22f8a07fa35: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
W20260929 10:53:15.878291 4693 tablet.cc:2369] T 09a5c9c3f7d94b5bad74b87caf975912 P 8ba6deb8d47147268bd65549332255ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:53:15.940913 4840 raft_consensus.cc:482] T 09a5c9c3f7d94b5bad74b87caf975912 P ee54fe9fec6d4197862fa22f8a07fa35 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:53:15.941104 4840 raft_consensus.cc:504] T 09a5c9c3f7d94b5bad74b87caf975912 P ee54fe9fec6d4197862fa22f8a07fa35 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba6deb8d47147268bd65549332255ba" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 39931 } } peers { permanent_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39941 } } peers { permanent_uuid: "6068e15b32f344179f1972d1fe7e3c52" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 44365 } }
I20260929 10:53:15.941601 4840 leader_election.cc:288] T 09a5c9c3f7d94b5bad74b87caf975912 P ee54fe9fec6d4197862fa22f8a07fa35 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ba6deb8d47147268bd65549332255ba (127.20.80.194:39931), 6068e15b32f344179f1972d1fe7e3c52 (127.20.80.193:44365)
I20260929 10:53:15.944840 4561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:15.941877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:57509 (local address 127.20.80.193:44365)
0929 10:53:15.941941 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:15.941943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:15.941954 (+ 11us) server_negotiation.cc:407] Connection header received
0929 10:53:15.942021 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:15.942021 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:15.942033 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:15.942113 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:53:15.942350 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.943091 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.944059 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.944459 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.944547 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:15.944557 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:15.944732 (+ 175us) server_negotiation.cc:299] Negotiation successful
0929 10:53:15.944773 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 10:53:15.944840 4827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:15.941839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:44365 (local address 127.20.80.195:57509)
0929 10:53:15.941898 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:15.941909 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:15.941956 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:15.942154 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:15.942155 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:15.942157 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:53:15.942252 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:15.942255 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.943201 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.943202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:15.943957 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:15.943959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.944545 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.944546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:15.944633 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:15.944637 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:53:15.944721 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 10:53:15.944764 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260929 10:53:15.945245 4501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09a5c9c3f7d94b5bad74b87caf975912" candidate_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6068e15b32f344179f1972d1fe7e3c52" is_pre_election: true
I20260929 10:53:15.945392 4501 raft_consensus.cc:2454] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee54fe9fec6d4197862fa22f8a07fa35 in term 0.
I20260929 10:53:15.945647 4711 leader_election.cc:302] T 09a5c9c3f7d94b5bad74b87caf975912 P ee54fe9fec6d4197862fa22f8a07fa35 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6068e15b32f344179f1972d1fe7e3c52, ee54fe9fec6d4197862fa22f8a07fa35; no voters:
I20260929 10:53:15.945749 4841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:15.942336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:39931 (local address 127.20.80.195:36411)
0929 10:53:15.942773 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:15.942785 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:15.942856 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:15.942997 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:15.942997 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:15.942999 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:53:15.943073 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:15.943075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.944009 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.944010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:15.944727 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:15.944729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.945489 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.945490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:15.945576 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:15.945581 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 10:53:15.945639 (+ 58us) client_negotiation.cc:240] Negotiation successful
0929 10:53:15.945676 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":97,"threads_started":1}
I20260929 10:53:15.945777 4831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:15.942578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:36411 (local address 127.20.80.194:39931)
0929 10:53:15.942626 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:15.942628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:15.942866 (+ 238us) server_negotiation.cc:407] Connection header received
0929 10:53:15.942904 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:15.942905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:15.942915 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:15.942991 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:53:15.943193 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.943889 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.944834 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.945399 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.945477 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:15.945484 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:15.945667 (+ 183us) server_negotiation.cc:299] Negotiation successful
0929 10:53:15.945710 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260929 10:53:15.945806 4840 raft_consensus.cc:2781] T 09a5c9c3f7d94b5bad74b87caf975912 P ee54fe9fec6d4197862fa22f8a07fa35 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:53:15.945854 4840 raft_consensus.cc:482] T 09a5c9c3f7d94b5bad74b87caf975912 P ee54fe9fec6d4197862fa22f8a07fa35 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:53:15.945884 4840 raft_consensus.cc:3037] T 09a5c9c3f7d94b5bad74b87caf975912 P ee54fe9fec6d4197862fa22f8a07fa35 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:53:15.946120 4646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09a5c9c3f7d94b5bad74b87caf975912" candidate_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ba6deb8d47147268bd65549332255ba" is_pre_election: true
I20260929 10:53:15.946290 4646 raft_consensus.cc:2454] T 09a5c9c3f7d94b5bad74b87caf975912 P 8ba6deb8d47147268bd65549332255ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee54fe9fec6d4197862fa22f8a07fa35 in term 0.
I20260929 10:53:15.946705 4840 raft_consensus.cc:504] T 09a5c9c3f7d94b5bad74b87caf975912 P ee54fe9fec6d4197862fa22f8a07fa35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba6deb8d47147268bd65549332255ba" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 39931 } } peers { permanent_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39941 } } peers { permanent_uuid: "6068e15b32f344179f1972d1fe7e3c52" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 44365 } }
I20260929 10:53:15.946868 4840 leader_election.cc:288] T 09a5c9c3f7d94b5bad74b87caf975912 P ee54fe9fec6d4197862fa22f8a07fa35 [CANDIDATE]: Term 1 election: Requested vote from peers 8ba6deb8d47147268bd65549332255ba (127.20.80.194:39931), 6068e15b32f344179f1972d1fe7e3c52 (127.20.80.193:44365)
I20260929 10:53:15.947011 4646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09a5c9c3f7d94b5bad74b87caf975912" candidate_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ba6deb8d47147268bd65549332255ba"
I20260929 10:53:15.947028 4501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09a5c9c3f7d94b5bad74b87caf975912" candidate_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6068e15b32f344179f1972d1fe7e3c52"
I20260929 10:53:15.947105 4501 raft_consensus.cc:3037] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:53:15.947118 4646 raft_consensus.cc:3037] T 09a5c9c3f7d94b5bad74b87caf975912 P 8ba6deb8d47147268bd65549332255ba [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:53:15.947835 4646 raft_consensus.cc:2454] T 09a5c9c3f7d94b5bad74b87caf975912 P 8ba6deb8d47147268bd65549332255ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee54fe9fec6d4197862fa22f8a07fa35 in term 1.
I20260929 10:53:15.948041 4714 leader_election.cc:302] T 09a5c9c3f7d94b5bad74b87caf975912 P ee54fe9fec6d4197862fa22f8a07fa35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ba6deb8d47147268bd65549332255ba, ee54fe9fec6d4197862fa22f8a07fa35; no voters:
I20260929 10:53:15.948179 4840 raft_consensus.cc:2781] T 09a5c9c3f7d94b5bad74b87caf975912 P ee54fe9fec6d4197862fa22f8a07fa35 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:53:15.948268 4501 raft_consensus.cc:2454] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee54fe9fec6d4197862fa22f8a07fa35 in term 1.
I20260929 10:53:15.948365 4840 raft_consensus.cc:686] T 09a5c9c3f7d94b5bad74b87caf975912 P ee54fe9fec6d4197862fa22f8a07fa35 [term 1 LEADER]: Becoming Leader. State: Replica: ee54fe9fec6d4197862fa22f8a07fa35, State: Running, Role: LEADER
I20260929 10:53:15.948457 4840 consensus_queue.cc:237] T 09a5c9c3f7d94b5bad74b87caf975912 P ee54fe9fec6d4197862fa22f8a07fa35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba6deb8d47147268bd65549332255ba" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 39931 } } peers { permanent_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39941 } } peers { permanent_uuid: "6068e15b32f344179f1972d1fe7e3c52" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 44365 } }
I20260929 10:53:15.949295 4363 catalog_manager.cc:5495] T 09a5c9c3f7d94b5bad74b87caf975912 P ee54fe9fec6d4197862fa22f8a07fa35 reported cstate change: term changed from 0 to 1, leader changed from <none> to ee54fe9fec6d4197862fa22f8a07fa35 (127.20.80.195). New cstate: current_term: 1 leader_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba6deb8d47147268bd65549332255ba" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 39931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6068e15b32f344179f1972d1fe7e3c52" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 44365 } health_report { overall_health: UNKNOWN } } }
I20260929 10:53:15.976191 4832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:15.974037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42508 (local address 127.20.80.195:39941)
0929 10:53:15.974091 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:15.974093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:15.974107 (+ 14us) server_negotiation.cc:407] Connection header received
0929 10:53:15.974139 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:15.974140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:15.974156 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:15.974210 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:53:15.974327 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.974777 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.975481 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:15.975657 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:15.975708 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:15.975870 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:53:15.975958 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:53:15.976018 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:15.976049 (+ 31us) server_negotiation.cc:299] Negotiation successful
0929 10:53:15.976066 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260929 10:53:15.977948 4646 raft_consensus.cc:1261] T 09a5c9c3f7d94b5bad74b87caf975912 P 8ba6deb8d47147268bd65549332255ba [term 1 FOLLOWER]: Refusing update from remote peer ee54fe9fec6d4197862fa22f8a07fa35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:53:15.977974 4501 raft_consensus.cc:1261] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [term 1 FOLLOWER]: Refusing update from remote peer ee54fe9fec6d4197862fa22f8a07fa35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:53:15.978289 4840 consensus_queue.cc:1035] T 09a5c9c3f7d94b5bad74b87caf975912 P ee54fe9fec6d4197862fa22f8a07fa35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6068e15b32f344179f1972d1fe7e3c52" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 44365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:53:15.978443 4842 consensus_queue.cc:1035] T 09a5c9c3f7d94b5bad74b87caf975912 P ee54fe9fec6d4197862fa22f8a07fa35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ba6deb8d47147268bd65549332255ba" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 39931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:53:15.981426 4847 mvcc.cc:205] Tried to move back new op lower bound from 7334621986723086336 to 7334621986605424640. Current Snapshot: MvccSnapshot[applied={T|T < 7334621986723086336}]
I20260929 10:53:15.982491 4362 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:59610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:53:15.982587 4362 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:59610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:53:15.983309 4362 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537: Sending DeleteTablet for 3 replicas of tablet 09a5c9c3f7d94b5bad74b87caf975912
I20260929 10:53:15.983594 4758 tablet_service.cc:1514] Processing DeleteTablet for tablet 09a5c9c3f7d94b5bad74b87caf975912 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:53:15 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:42506
I20260929 10:53:15.983649 4487 tablet_service.cc:1514] Processing DeleteTablet for tablet 09a5c9c3f7d94b5bad74b87caf975912 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:53:15 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:35258
I20260929 10:53:15.983673 4626 tablet_service.cc:1514] Processing DeleteTablet for tablet 09a5c9c3f7d94b5bad74b87caf975912 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:53:15 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:44752
I20260929 10:53:15.983740 4562 tablet_replica.cc:330] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52: stopping tablet replica
I20260929 10:53:15.983786 4562 raft_consensus.cc:2229] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:53:15.983862 4562 raft_consensus.cc:2258] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:53:15.983922 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 4402
I20260929 10:53:15.984141 4562 ts_tablet_manager.cc:1905] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:53:15.983875 4855 tablet_replica.cc:330] T 09a5c9c3f7d94b5bad74b87caf975912 P 8ba6deb8d47147268bd65549332255ba: stopping tablet replica
I20260929 10:53:15.984486 4855 raft_consensus.cc:2229] T 09a5c9c3f7d94b5bad74b87caf975912 P 8ba6deb8d47147268bd65549332255ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:53:15.984582 4855 raft_consensus.cc:2258] T 09a5c9c3f7d94b5bad74b87caf975912 P 8ba6deb8d47147268bd65549332255ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:53:15.985011 4855 ts_tablet_manager.cc:1905] T 09a5c9c3f7d94b5bad74b87caf975912 P 8ba6deb8d47147268bd65549332255ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:53:15.986099 4854 tablet_replica.cc:330] T 09a5c9c3f7d94b5bad74b87caf975912 P ee54fe9fec6d4197862fa22f8a07fa35: stopping tablet replica
I20260929 10:53:15.986231 4854 raft_consensus.cc:2229] T 09a5c9c3f7d94b5bad74b87caf975912 P ee54fe9fec6d4197862fa22f8a07fa35 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:53:15.986367 4854 raft_consensus.cc:2258] T 09a5c9c3f7d94b5bad74b87caf975912 P ee54fe9fec6d4197862fa22f8a07fa35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:53:15.986721 4855 ts_tablet_manager.cc:1918] T 09a5c9c3f7d94b5bad74b87caf975912 P 8ba6deb8d47147268bd65549332255ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:53:15.986796 4855 log.cc:1196] T 09a5c9c3f7d94b5bad74b87caf975912 P 8ba6deb8d47147268bd65549332255ba: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/wal/wals/09a5c9c3f7d94b5bad74b87caf975912
I20260929 10:53:15.986754 4854 ts_tablet_manager.cc:1905] T 09a5c9c3f7d94b5bad74b87caf975912 P ee54fe9fec6d4197862fa22f8a07fa35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:53:15.987082 4855 ts_tablet_manager.cc:1939] T 09a5c9c3f7d94b5bad74b87caf975912 P 8ba6deb8d47147268bd65549332255ba: Deleting consensus metadata
I20260929 10:53:15.987473 4350 catalog_manager.cc:4841] TS 8ba6deb8d47147268bd65549332255ba (127.20.80.194:39931): tablet 09a5c9c3f7d94b5bad74b87caf975912 (table test-table [id=d5b4025e0aad4de7836d8ad752d6edb9]) successfully deleted
I20260929 10:53:15.989212 4854 ts_tablet_manager.cc:1918] T 09a5c9c3f7d94b5bad74b87caf975912 P ee54fe9fec6d4197862fa22f8a07fa35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:53:15.989347 4854 log.cc:1196] T 09a5c9c3f7d94b5bad74b87caf975912 P ee54fe9fec6d4197862fa22f8a07fa35: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/wal/wals/09a5c9c3f7d94b5bad74b87caf975912
I20260929 10:53:15.989674 4854 ts_tablet_manager.cc:1939] T 09a5c9c3f7d94b5bad74b87caf975912 P ee54fe9fec6d4197862fa22f8a07fa35: Deleting consensus metadata
I20260929 10:53:15.990114 4349 catalog_manager.cc:4841] TS ee54fe9fec6d4197862fa22f8a07fa35 (127.20.80.195:39941): tablet 09a5c9c3f7d94b5bad74b87caf975912 (table test-table [id=d5b4025e0aad4de7836d8ad752d6edb9]) successfully deleted
W20260929 10:53:15.990993 4347 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.80.193:44365 (error 108)
I20260929 10:53:15.991384 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 4557
I20260929 10:53:15.991410 4833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:15.991304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:44365 (local address 127.0.0.1:35272)
0929 10:53:15.991345 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:15.991385 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.193:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260929 10:53:15.991524 4347 catalog_manager.cc:4584] TS 6068e15b32f344179f1972d1fe7e3c52 (127.20.80.193:44365): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 09a5c9c3f7d94b5bad74b87caf975912: Network error: Client connection negotiation failed: client connection to 127.20.80.193:44365: connect: Connection refused (error 111)
I20260929 10:53:15.996474 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 4695
I20260929 10:53:16.001185 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 4330
I20260929 10:53:36.006675 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:39195
--webserver_interface=127.20.80.254
--webserver_port=42113
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:46069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:39195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:53:36.086719 4857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:53:36.086933 4857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:53:36.086956 4857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:53:36.088024 4857 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 10:53:36.088056 4857 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 10:53:36.088069 4857 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 10:53:36.088081 4857 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:53:36.088094 4857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:53:36.088107 4857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:53:36.088120 4857 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:53:36.088137 4857 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:53:36.089574 4857 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:46069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:39195
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:39195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=42113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4857
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:53:36.089763 4857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:53:36.089943 4857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:53:36.094900 4863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:36.094878 4862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:36.094846 4865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:53:36.094918 4857 server_base.cc:1034] running on GCE node
I20260929 10:53:36.095450 4857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:53:36.095786 4857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:53:36.097041 4857 hybrid_clock.cc:648] HybridClock initialized: now 1790679216096975 us; error 36 us; skew 500 ppm
Sep 29 10:53:36 dist-test-slave-vqch krb5kdc[3778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679216, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:53:36.104329 4857 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:53:36.104823 4857 webserver.cc:460] Webserver started at http://127.20.80.254:42113/ using document root <none> and password file <none>
I20260929 10:53:36.105118 4857 fs_manager.cc:359] Metadata directory not provided
I20260929 10:53:36.105189 4857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:53:36.106592 4857 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:53:36.107292 4872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:36.107470 4857 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:53:36.107566 4857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "a1657adc941042819338bbf51f53f537"
format_stamp: "Formatted at 2026-09-29 10:52:54 on dist-test-slave-vqch"
server_key: "baeaf904359347022e9a2a8d6e8ec9ad"
server_key_iv: "1577570795ac1af689bf3c04ebc2732e"
server_key_version: "encryptionkey@0"
I20260929 10:53:36.107951 4857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:53:36.125201 4857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:53:36.127561 4857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:53:36.127727 4857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:53:36.131976 4857 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:39195
I20260929 10:53:36.132022 4925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:39195 every 8 connection(s)
I20260929 10:53:36.132434 4857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:53:36.133962 4926 sys_catalog.cc:263] Verifying existing consensus state
I20260929 10:53:36.134560 4926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537: Bootstrap starting.
I20260929 10:53:36.141903 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 4857
I20260929 10:53:36.142264 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.193:44365
--local_ip_for_outbound_sockets=127.20.80.193
--tserver_master_addrs=127.20.80.254:39195
--webserver_port=36339
--webserver_interface=127.20.80.193
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:46069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:53:36.143895 4926 log.cc:824] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537: Log is configured to *not* fsync() on all Append() calls
I20260929 10:53:36.146476 4926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 10:53:36.146793 4926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537: Bootstrap complete.
I20260929 10:53:36.149065 4926 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1657adc941042819338bbf51f53f537" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 39195 } }
I20260929 10:53:36.149312 4926 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1657adc941042819338bbf51f53f537, State: Initialized, Role: FOLLOWER
I20260929 10:53:36.149479 4926 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1657adc941042819338bbf51f53f537" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 39195 } }
I20260929 10:53:36.149541 4926 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:53:36.149603 4926 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:53:36.149663 4926 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [term 2 FOLLOWER]: Advancing to term 3
I20260929 10:53:36.150665 4926 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1657adc941042819338bbf51f53f537" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 39195 } }
I20260929 10:53:36.150754 4926 leader_election.cc:302] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1657adc941042819338bbf51f53f537; no voters:
I20260929 10:53:36.150964 4926 leader_election.cc:288] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260929 10:53:36.151001 4930 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [term 3 FOLLOWER]: Leader election won for term 3
I20260929 10:53:36.151113 4930 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [term 3 LEADER]: Becoming Leader. State: Replica: a1657adc941042819338bbf51f53f537, State: Running, Role: LEADER
I20260929 10:53:36.151248 4926 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:53:36.151273 4930 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1657adc941042819338bbf51f53f537" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 39195 } }
I20260929 10:53:36.151609 4931 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a1657adc941042819338bbf51f53f537" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1657adc941042819338bbf51f53f537" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 39195 } } }
I20260929 10:53:36.151681 4931 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [sys.catalog]: This master's current role is: LEADER
I20260929 10:53:36.151974 4932 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1657adc941042819338bbf51f53f537. Latest consensus state: current_term: 3 leader_uuid: "a1657adc941042819338bbf51f53f537" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1657adc941042819338bbf51f53f537" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 39195 } } }
I20260929 10:53:36.152055 4932 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537 [sys.catalog]: This master's current role is: LEADER
I20260929 10:53:36.152331 4935 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:53:36.153174 4935 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:53:36.153327 4935 catalog_manager.cc:1259] Loaded cluster ID: e51b1a7e7f214bc48fb8541e0ad3972e
I20260929 10:53:36.153543 4935 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:53:36.154837 4935 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:53:36.155071 4935 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537: Loaded TSK: 0
I20260929 10:53:36.155143 4935 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a1657adc941042819338bbf51f53f537: Loaded TSK: 1
F20260929 10:53:36.155539 4935 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f11ae9019ad google::LogMessage::Fail() at ??:0
@ 0x7f11ae90488a google::LogMessage::SendToLog() at ??:0
@ 0x7f11ae9014de google::LogMessage::Flush() at ??:0
@ 0x7f11ae902a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f11b106e104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f11b106bd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f11b685b1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f11b685b48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f11b685b594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f11b6821c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f11b682bad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f11b07cf0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f11b07c4954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f11b16286db start_thread at ??:0
@ 0x7f11ad3ff61f clone at ??:0
W20260929 10:53:36.245221 4928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:53:36.245411 4928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:53:36.245434 4928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:53:36.246619 4928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:53:36.246697 4928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:53:36.246724 4928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.193
I20260929 10:53:36.248168 4928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:46069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.193:44365
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.80.193
--webserver_port=36339
--tserver_master_addrs=127.20.80.254:39195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.193
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:53:36.248394 4928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:53:36.248637 4928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:53:36.253592 4968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:36.253567 4969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:53:36.253739 4928 server_base.cc:1034] running on GCE node
W20260929 10:53:36.253742 4971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:53:36.254051 4928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:53:36.254352 4928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:53:36.255519 4928 hybrid_clock.cc:648] HybridClock initialized: now 1790679216255506 us; error 27 us; skew 500 ppm
Sep 29 10:53:36 dist-test-slave-vqch krb5kdc[3778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679216, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:53:36.258051 4928 init.cc:377] Logged in from keytab as kudu/127.20.80.193@KRBTEST.COM (short username kudu)
I20260929 10:53:36.258522 4928 webserver.cc:460] Webserver started at http://127.20.80.193:36339/ using document root <none> and password file <none>
I20260929 10:53:36.258759 4928 fs_manager.cc:359] Metadata directory not provided
I20260929 10:53:36.258805 4928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:53:36.260147 4928 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:53:36.260912 4978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:36.261104 4928 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:53:36.261209 4928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/wal
uuid: "6068e15b32f344179f1972d1fe7e3c52"
format_stamp: "Formatted at 2026-09-29 10:52:54 on dist-test-slave-vqch"
server_key: "a243feabe40acd714bdb8f24f725bb07"
server_key_iv: "e19c45bb6b848a94639cac5062fab8ae"
server_key_version: "encryptionkey@0"
I20260929 10:53:36.261533 4928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:53:36.276188 4928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:53:36.278687 4928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:53:36.278834 4928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:53:36.279114 4928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:53:36.279606 4985 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 10:53:36.281284 4928 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 10:53:36.281363 4928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260929 10:53:36.281424 4928 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 10:53:36.282196 4928 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 10:53:36.282253 4928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260929 10:53:36.282332 4985 tablet_bootstrap.cc:492] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52: Bootstrap starting.
I20260929 10:53:36.284202 4985 log.cc:824] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52: Log is configured to *not* fsync() on all Append() calls
I20260929 10:53:36.287514 4985 tablet_bootstrap.cc:492] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 10:53:36.287955 4985 tablet_bootstrap.cc:492] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52: Bootstrap complete.
I20260929 10:53:36.288160 4985 ts_tablet_manager.cc:1397] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260929 10:53:36.288635 4928 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.193:44365
I20260929 10:53:36.288748 5093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.193:44365 every 8 connection(s)
I20260929 10:53:36.289088 4928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
I20260929 10:53:36.290210 4985 raft_consensus.cc:348] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba6deb8d47147268bd65549332255ba" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 39931 } } peers { permanent_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39941 } } peers { permanent_uuid: "6068e15b32f344179f1972d1fe7e3c52" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 44365 } }
I20260929 10:53:36.290520 4985 raft_consensus.cc:729] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6068e15b32f344179f1972d1fe7e3c52, State: Initialized, Role: FOLLOWER
I20260929 10:53:36.290692 4985 consensus_queue.cc:260] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba6deb8d47147268bd65549332255ba" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 39931 } } peers { permanent_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39941 } } peers { permanent_uuid: "6068e15b32f344179f1972d1fe7e3c52" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 44365 } }
I20260929 10:53:36.290944 4985 ts_tablet_manager.cc:1428] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 10:53:36.298105 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 4928
I20260929 10:53:36.298404 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.194:39931
--local_ip_for_outbound_sockets=127.20.80.194
--tserver_master_addrs=127.20.80.254:39195
--webserver_port=45783
--webserver_interface=127.20.80.194
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:46069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:53:36.344022 5096 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:53:36.289796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.193:50877)
0929 10:53:36.289994 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:36.290043 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:36.290218 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:36.343816 (+ 53598us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":83,"threads_started":1}
W20260929 10:53:36.344329 5094 heartbeater.cc:644] Failed to heartbeat to 127.20.80.254:39195 (0 consecutive failures): Network error: Failed to ping master at 127.20.80.254:39195: Client connection negotiation failed: client connection to 127.20.80.254:39195: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 10:53:36.344710 5096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:36.344589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.193:39643)
0929 10:53:36.344647 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:36.344689 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260929 10:53:36.345129 5096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:36.345045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.193:53753)
0929 10:53:36.345082 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:36.345108 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260929 10:53:36.345278 5094 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 10:53:36.382061 5098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:53:36.382262 5098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:53:36.382287 5098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:53:36.383461 5098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:53:36.383504 5098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:53:36.383531 5098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.194
I20260929 10:53:36.384972 5098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:46069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.194:39931
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.80.194
--webserver_port=45783
--tserver_master_addrs=127.20.80.254:39195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.194
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:53:36.385239 5098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:53:36.385478 5098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:53:36.390651 5103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:36.390625 5106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:36.390633 5104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:53:36.390717 5098 server_base.cc:1034] running on GCE node
I20260929 10:53:36.391211 5098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:53:36.391451 5098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:53:36.392597 5098 hybrid_clock.cc:648] HybridClock initialized: now 1790679216392576 us; error 37 us; skew 500 ppm
Sep 29 10:53:36 dist-test-slave-vqch krb5kdc[3778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679216, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:53:36.395357 5098 init.cc:377] Logged in from keytab as kudu/127.20.80.194@KRBTEST.COM (short username kudu)
I20260929 10:53:36.395828 5098 webserver.cc:460] Webserver started at http://127.20.80.194:45783/ using document root <none> and password file <none>
I20260929 10:53:36.396077 5098 fs_manager.cc:359] Metadata directory not provided
I20260929 10:53:36.396157 5098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:53:36.397576 5098 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260929 10:53:36.398144 5113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:36.398310 5098 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:36.398413 5098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/wal
uuid: "8ba6deb8d47147268bd65549332255ba"
format_stamp: "Formatted at 2026-09-29 10:52:54 on dist-test-slave-vqch"
server_key: "b099fc61df5a00b45a75c1142b1fb82f"
server_key_iv: "748e729b9178b339e4e0e5ef01d61646"
server_key_version: "encryptionkey@0"
I20260929 10:53:36.398757 5098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:53:36.410476 5098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:53:36.412305 5098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:53:36.412484 5098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:53:36.412711 5098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:53:36.413089 5098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:53:36.413146 5098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:36.413188 5098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:53:36.413214 5098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:36.419260 5098 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.194:39931
I20260929 10:53:36.419305 5229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.194:39931 every 8 connection(s)
I20260929 10:53:36.419644 5098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
I20260929 10:53:36.420704 5232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:36.420298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.194:51523)
0929 10:53:36.420450 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:36.420561 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
W20260929 10:53:36.421164 5230 heartbeater.cc:644] Failed to heartbeat to 127.20.80.254:39195 (0 consecutive failures): Network error: Failed to ping master at 127.20.80.254:39195: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
I20260929 10:53:36.421500 5232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:36.421409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.194:41859)
0929 10:53:36.421457 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:36.421483 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260929 10:53:36.421804 5232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:36.421725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.194:55133)
0929 10:53:36.421768 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:36.421788 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260929 10:53:36.421933 5230 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 10:53:36.423625 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 5098
I20260929 10:53:36.423868 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.195:39941
--local_ip_for_outbound_sockets=127.20.80.195
--tserver_master_addrs=127.20.80.254:39195
--webserver_port=39871
--webserver_interface=127.20.80.195
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:46069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:53:36.509420 5233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:53:36.509611 5233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:53:36.509634 5233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:53:36.510697 5233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:53:36.510730 5233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:53:36.510756 5233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.195
I20260929 10:53:36.512080 5233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:46069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.195:39941
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.80.195
--webserver_port=39871
--tserver_master_addrs=127.20.80.254:39195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.195
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:53:36.512250 5233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:53:36.512432 5233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:53:36.517338 5239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:36.517313 5238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:36.517298 5241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:53:36.517465 5233 server_base.cc:1034] running on GCE node
I20260929 10:53:36.517694 5233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:53:36.517927 5233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:53:36.519085 5233 hybrid_clock.cc:648] HybridClock initialized: now 1790679216519073 us; error 27 us; skew 500 ppm
Sep 29 10:53:36 dist-test-slave-vqch krb5kdc[3778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679216, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:53:36.521525 5233 init.cc:377] Logged in from keytab as kudu/127.20.80.195@KRBTEST.COM (short username kudu)
I20260929 10:53:36.522007 5233 webserver.cc:460] Webserver started at http://127.20.80.195:39871/ using document root <none> and password file <none>
I20260929 10:53:36.522228 5233 fs_manager.cc:359] Metadata directory not provided
I20260929 10:53:36.522295 5233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:53:36.523670 5233 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:53:36.524269 5248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:36.524437 5233 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:53:36.524534 5233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/wal
uuid: "ee54fe9fec6d4197862fa22f8a07fa35"
format_stamp: "Formatted at 2026-09-29 10:52:55 on dist-test-slave-vqch"
server_key: "9c6042a48842f0a37e6165c171fd5bf8"
server_key_iv: "93102fa2138c42adf8f2c705798eb20b"
server_key_version: "encryptionkey@0"
I20260929 10:53:36.524896 5233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:53:36.534519 5233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:53:36.536263 5233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:53:36.536367 5233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:53:36.536530 5233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:53:36.536836 5233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:53:36.536867 5233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:36.536890 5233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:53:36.536937 5233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:36.543023 5233 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.195:39941
I20260929 10:53:36.543066 5363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.195:39941 every 8 connection(s)
I20260929 10:53:36.543387 5233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
I20260929 10:53:36.544358 5366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:36.543996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.195:47861)
0929 10:53:36.544149 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:36.544225 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":73,"threads_started":1}
W20260929 10:53:36.544806 5364 heartbeater.cc:644] Failed to heartbeat to 127.20.80.254:39195 (0 consecutive failures): Network error: Failed to ping master at 127.20.80.254:39195: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
I20260929 10:53:36.545145 5366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:36.545052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.195:42555)
0929 10:53:36.545101 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:36.545127 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260929 10:53:36.545527 5366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:36.545395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.195:43529)
0929 10:53:36.545475 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:36.545505 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260929 10:53:36.545763 5364 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 10:53:36.548784 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 5233
I20260929 10:53:37.345890 5368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:37.345658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.193:49911)
0929 10:53:37.345783 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:37.345826 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
I20260929 10:53:37.422492 5369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:37.422293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.194:39799)
0929 10:53:37.422407 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:37.422443 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":55,"threads_started":1}
I20260929 10:53:37.488219 5370 raft_consensus.cc:482] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:53:37.488343 5370 raft_consensus.cc:504] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba6deb8d47147268bd65549332255ba" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 39931 } } peers { permanent_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39941 } } peers { permanent_uuid: "6068e15b32f344179f1972d1fe7e3c52" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 44365 } }
I20260929 10:53:37.488682 5370 leader_election.cc:288] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ba6deb8d47147268bd65549332255ba (127.20.80.194:39931), ee54fe9fec6d4197862fa22f8a07fa35 (127.20.80.195:39941)
Sep 29 10:53:37 dist-test-slave-vqch krb5kdc[3778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679216, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for kudu/127.20.80.195@KRBTEST.COM
Sep 29 10:53:37 dist-test-slave-vqch krb5kdc[3778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679216, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for kudu/127.20.80.194@KRBTEST.COM
I20260929 10:53:37.499284 5368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:37.488785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:39931 (local address 127.20.80.193:54357)
0929 10:53:37.488846 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:37.488859 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:37.488922 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:37.489888 (+ 966us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:37.489890 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:37.490294 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:53:37.490519 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:37.490526 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:37.491707 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:37.491708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:37.492526 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:37.492529 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:37.493197 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:37.493198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:37.493324 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:37.493973 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:53:37.493981 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:53:37.495900 (+ 1919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:53:37.497856 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:53:37.497857 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:53:37.497857 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:53:37.498044 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:53:37.498311 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:53:37.498311 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:53:37.498312 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:53:37.498376 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:53:37.498750 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:53:37.498757 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:53:37.498885 (+ 128us) client_negotiation.cc:769] Sending connection context
0929 10:53:37.498994 (+ 109us) client_negotiation.cc:240] Negotiation successful
0929 10:53:37.499181 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":4}
I20260929 10:53:37.499295 5372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:37.488944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:54357 (local address 127.20.80.194:39931)
0929 10:53:37.489143 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:37.489161 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:37.489209 (+ 48us) server_negotiation.cc:407] Connection header received
0929 10:53:37.489668 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:37.489670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:37.489709 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:37.489861 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:53:37.490651 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:37.491591 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:37.492631 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:37.493095 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:37.493240 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:37.496007 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:53:37.496010 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:53:37.496011 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:53:37.496023 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:53:37.497751 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:37.498140 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:37.498143 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:37.498143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:37.498234 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:37.498441 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:37.498442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:37.498442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:37.498670 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:53:37.498755 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:37.499048 (+ 293us) server_negotiation.cc:299] Negotiation successful
0929 10:53:37.499181 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":90,"threads_started":1}
I20260929 10:53:37.499301 5373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:37.489013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:38805 (local address 127.20.80.195:39941)
0929 10:53:37.489169 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:37.489177 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:37.489195 (+ 18us) server_negotiation.cc:407] Connection header received
0929 10:53:37.489620 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:37.489622 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:37.489683 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:37.489832 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:53:37.490809 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:37.491666 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:37.492628 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:37.493117 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:37.493246 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:37.495767 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:53:37.495774 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:53:37.495774 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:53:37.495789 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:53:37.497613 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:37.498084 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:37.498087 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:37.498087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:37.498178 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:37.498434 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:37.498435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:37.498435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:37.498667 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:53:37.498761 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:37.499049 (+ 288us) server_negotiation.cc:299] Negotiation successful
0929 10:53:37.499174 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":47,"threads_started":1}
I20260929 10:53:37.499416 5371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:37.488874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:39941 (local address 127.20.80.193:38805)
0929 10:53:37.489013 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:37.489021 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:37.489063 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:37.489888 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:37.489890 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:37.490292 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:53:37.490683 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:37.490685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:37.491765 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:37.491766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:37.492529 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:37.492532 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:37.493214 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:37.493215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:37.493325 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:37.494023 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:53:37.494028 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:53:37.495516 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:53:37.497717 (+ 2201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:53:37.497724 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:53:37.497726 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:53:37.497985 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:53:37.498257 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:53:37.498258 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:53:37.498258 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:53:37.498364 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:53:37.498760 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:53:37.498761 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:53:37.498883 (+ 122us) client_negotiation.cc:769] Sending connection context
0929 10:53:37.498994 (+ 111us) client_negotiation.cc:240] Negotiation successful
0929 10:53:37.499262 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":207,"thread_start_us":55,"threads_started":1}
W20260929 10:53:37.500720 4982 leader_election.cc:341] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ba6deb8d47147268bd65549332255ba (127.20.80.194:39931): Not found: Tablet not found: 09a5c9c3f7d94b5bad74b87caf975912
W20260929 10:53:37.500857 4981 leader_election.cc:341] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee54fe9fec6d4197862fa22f8a07fa35 (127.20.80.195:39941): Not found: Tablet not found: 09a5c9c3f7d94b5bad74b87caf975912
I20260929 10:53:37.500919 4981 leader_election.cc:302] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6068e15b32f344179f1972d1fe7e3c52; no voters: 8ba6deb8d47147268bd65549332255ba, ee54fe9fec6d4197862fa22f8a07fa35
I20260929 10:53:37.501049 5370 raft_consensus.cc:2726] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 10:53:37.546379 5374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:37.546136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.195:34333)
0929 10:53:37.546288 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:37.546339 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":56,"threads_started":1}
I20260929 10:53:38.346623 5375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:38.346390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.193:43237)
0929 10:53:38.346531 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:38.346578 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":59,"threads_started":1}
I20260929 10:53:38.423262 5376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:38.422926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.194:40369)
0929 10:53:38.423193 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:38.423224 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":80,"threads_started":1}
I20260929 10:53:38.547113 5377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:38.546881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.195:53187)
0929 10:53:38.547015 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:38.547061 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":61,"threads_started":1}
I20260929 10:53:39.347338 5378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:39.347119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.193:43825)
0929 10:53:39.347247 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:39.347291 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":63,"threads_started":1}
I20260929 10:53:39.424002 5379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:39.423771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.194:44515)
0929 10:53:39.423924 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:39.423960 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":77,"threads_started":1}
I20260929 10:53:39.547974 5380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:39.547709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.195:36111)
0929 10:53:39.547868 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:39.547910 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":75,"threads_started":1}
I20260929 10:53:39.599004 5381 raft_consensus.cc:482] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:53:39.599097 5381 raft_consensus.cc:504] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba6deb8d47147268bd65549332255ba" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 39931 } } peers { permanent_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39941 } } peers { permanent_uuid: "6068e15b32f344179f1972d1fe7e3c52" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 44365 } }
I20260929 10:53:39.599308 5381 leader_election.cc:288] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ba6deb8d47147268bd65549332255ba (127.20.80.194:39931), ee54fe9fec6d4197862fa22f8a07fa35 (127.20.80.195:39941)
W20260929 10:53:39.599809 4981 leader_election.cc:341] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee54fe9fec6d4197862fa22f8a07fa35 (127.20.80.195:39941): Not found: Tablet not found: 09a5c9c3f7d94b5bad74b87caf975912
W20260929 10:53:39.599888 4982 leader_election.cc:341] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ba6deb8d47147268bd65549332255ba (127.20.80.194:39931): Not found: Tablet not found: 09a5c9c3f7d94b5bad74b87caf975912
I20260929 10:53:39.599995 4982 leader_election.cc:302] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6068e15b32f344179f1972d1fe7e3c52; no voters: 8ba6deb8d47147268bd65549332255ba, ee54fe9fec6d4197862fa22f8a07fa35
I20260929 10:53:39.600103 5381 raft_consensus.cc:2726] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 10:53:40.348070 5382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:40.347865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.193:42733)
0929 10:53:40.347987 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:40.348024 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":62,"threads_started":1}
I20260929 10:53:40.424885 5383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:40.424599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.194:39843)
0929 10:53:40.424794 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:40.424838 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":87,"threads_started":1}
I20260929 10:53:40.548609 5384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:40.548403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.195:50647)
0929 10:53:40.548530 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:40.548569 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":55,"threads_started":1}
I20260929 10:53:41.348851 5385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:41.348627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.193:60915)
0929 10:53:41.348761 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:41.348801 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":69,"threads_started":1}
I20260929 10:53:41.425566 5386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:41.425374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.194:35497)
0929 10:53:41.425494 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:41.425529 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":54,"threads_started":1}
I20260929 10:53:41.549305 5387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:41.549105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.195:54303)
0929 10:53:41.549224 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:41.549260 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":57,"threads_started":1}
I20260929 10:53:42.285157 5388 raft_consensus.cc:482] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:53:42.285281 5388 raft_consensus.cc:504] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba6deb8d47147268bd65549332255ba" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 39931 } } peers { permanent_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39941 } } peers { permanent_uuid: "6068e15b32f344179f1972d1fe7e3c52" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 44365 } }
I20260929 10:53:42.285504 5388 leader_election.cc:288] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ba6deb8d47147268bd65549332255ba (127.20.80.194:39931), ee54fe9fec6d4197862fa22f8a07fa35 (127.20.80.195:39941)
W20260929 10:53:42.285929 4981 leader_election.cc:341] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee54fe9fec6d4197862fa22f8a07fa35 (127.20.80.195:39941): Not found: Tablet not found: 09a5c9c3f7d94b5bad74b87caf975912
W20260929 10:53:42.286051 4982 leader_election.cc:341] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ba6deb8d47147268bd65549332255ba (127.20.80.194:39931): Not found: Tablet not found: 09a5c9c3f7d94b5bad74b87caf975912
I20260929 10:53:42.286092 4982 leader_election.cc:302] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6068e15b32f344179f1972d1fe7e3c52; no voters: 8ba6deb8d47147268bd65549332255ba, ee54fe9fec6d4197862fa22f8a07fa35
I20260929 10:53:42.286216 5388 raft_consensus.cc:2726] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 10:53:42.349756 5389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:42.349496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.193:34207)
0929 10:53:42.349666 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:42.349707 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":78,"threads_started":1}
I20260929 10:53:42.426255 5390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:42.426024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.194:39059)
0929 10:53:42.426171 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:42.426211 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":62,"threads_started":1}
I20260929 10:53:42.550115 5391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:42.549879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.195:39115)
0929 10:53:42.550021 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:42.550069 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":62,"threads_started":1}
I20260929 10:53:43.350539 5392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:43.350310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.193:45991)
0929 10:53:43.350450 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:43.350493 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":69,"threads_started":1}
I20260929 10:53:43.426996 5393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:43.426730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.194:50931)
0929 10:53:43.426902 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:43.426948 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":63,"threads_started":1}
I20260929 10:53:43.550868 5394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:43.550621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.195:38791)
0929 10:53:43.550765 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:43.550811 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":74,"threads_started":1}
I20260929 10:53:44.351363 5395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:44.351100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.193:41837)
0929 10:53:44.351270 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:44.351314 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":63,"threads_started":1}
I20260929 10:53:44.427868 5396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:44.427587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.194:38073)
0929 10:53:44.427752 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:44.427813 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":119,"threads_started":1}
I20260929 10:53:44.551540 5397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:44.551324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.195:36773)
0929 10:53:44.551458 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:44.551496 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":72,"threads_started":1}
I20260929 10:53:45.352116 5398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:45.351858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.193:32875)
0929 10:53:45.352011 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:45.352067 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":60,"threads_started":1}
I20260929 10:53:45.428618 5399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:45.428326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.194:43391)
0929 10:53:45.428517 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:45.428563 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":103,"threads_started":1}
I20260929 10:53:45.552300 5400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:45.552035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.195:56803)
0929 10:53:45.552198 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:45.552254 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":73,"threads_started":1}
I20260929 10:53:45.704952 5401 raft_consensus.cc:482] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:53:45.705116 5401 raft_consensus.cc:504] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba6deb8d47147268bd65549332255ba" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 39931 } } peers { permanent_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39941 } } peers { permanent_uuid: "6068e15b32f344179f1972d1fe7e3c52" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 44365 } }
I20260929 10:53:45.705277 5401 leader_election.cc:288] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ba6deb8d47147268bd65549332255ba (127.20.80.194:39931), ee54fe9fec6d4197862fa22f8a07fa35 (127.20.80.195:39941)
W20260929 10:53:45.705693 4982 leader_election.cc:341] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ba6deb8d47147268bd65549332255ba (127.20.80.194:39931): Not found: Tablet not found: 09a5c9c3f7d94b5bad74b87caf975912
W20260929 10:53:45.705760 4981 leader_election.cc:341] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee54fe9fec6d4197862fa22f8a07fa35 (127.20.80.195:39941): Not found: Tablet not found: 09a5c9c3f7d94b5bad74b87caf975912
I20260929 10:53:45.705787 4982 leader_election.cc:302] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6068e15b32f344179f1972d1fe7e3c52; no voters: 8ba6deb8d47147268bd65549332255ba, ee54fe9fec6d4197862fa22f8a07fa35
I20260929 10:53:45.705869 5401 raft_consensus.cc:2726] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 10:53:46.353077 5402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:46.352799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.193:46177)
0929 10:53:46.352967 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:46.353029 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":84,"threads_started":1}
I20260929 10:53:46.429400 5403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:46.429121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.194:36913)
0929 10:53:46.429287 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:46.429323 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":75,"threads_started":1}
I20260929 10:53:46.553306 5404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:46.553051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.195:39647)
0929 10:53:46.553216 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:46.553259 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":74,"threads_started":1}
I20260929 10:53:47.353838 5405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:47.353568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.193:43043)
0929 10:53:47.353751 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:47.353790 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":111,"threads_started":1}
I20260929 10:53:47.430025 5406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:47.429820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.194:46007)
0929 10:53:47.429942 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:47.429977 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":57,"threads_started":1}
I20260929 10:53:47.554127 5407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:47.553825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.195:44845)
0929 10:53:47.554011 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:47.554048 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":65,"threads_started":1}
I20260929 10:53:48.354560 5408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:48.354343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.193:38181)
0929 10:53:48.354471 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:48.354510 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":61,"threads_started":1}
I20260929 10:53:48.428299 5409 raft_consensus.cc:482] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:53:48.428416 5409 raft_consensus.cc:504] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba6deb8d47147268bd65549332255ba" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 39931 } } peers { permanent_uuid: "ee54fe9fec6d4197862fa22f8a07fa35" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 39941 } } peers { permanent_uuid: "6068e15b32f344179f1972d1fe7e3c52" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 44365 } }
I20260929 10:53:48.428647 5409 leader_election.cc:288] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ba6deb8d47147268bd65549332255ba (127.20.80.194:39931), ee54fe9fec6d4197862fa22f8a07fa35 (127.20.80.195:39941)
W20260929 10:53:48.429016 4982 leader_election.cc:341] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ba6deb8d47147268bd65549332255ba (127.20.80.194:39931): Not found: Tablet not found: 09a5c9c3f7d94b5bad74b87caf975912
W20260929 10:53:48.429132 4981 leader_election.cc:341] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee54fe9fec6d4197862fa22f8a07fa35 (127.20.80.195:39941): Not found: Tablet not found: 09a5c9c3f7d94b5bad74b87caf975912
I20260929 10:53:48.429203 4981 leader_election.cc:302] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6068e15b32f344179f1972d1fe7e3c52; no voters: 8ba6deb8d47147268bd65549332255ba, ee54fe9fec6d4197862fa22f8a07fa35
I20260929 10:53:48.429301 5409 raft_consensus.cc:2726] T 09a5c9c3f7d94b5bad74b87caf975912 P 6068e15b32f344179f1972d1fe7e3c52 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 10:53:48.430737 5410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:48.430538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.194:35617)
0929 10:53:48.430662 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:48.430697 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":52,"threads_started":1}
I20260929 10:53:48.554773 5411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:48.554567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.195:40205)
0929 10:53:48.554684 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:48.554727 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260929 10:53:49.355381 5412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:49.355141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.193:53333)
0929 10:53:49.355294 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:49.355332 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":89,"threads_started":1}
I20260929 10:53:49.431392 5413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:49.431191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.194:45207)
0929 10:53:49.431312 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:49.431348 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
I20260929 10:53:49.555550 5414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:49.555274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.195:59993)
0929 10:53:49.555441 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:49.555485 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":85,"threads_started":1}
I20260929 10:53:50.356081 5415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:50.355859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.193:48421)
0929 10:53:50.355998 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:50.356034 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":64,"threads_started":1}
I20260929 10:53:50.432147 5416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:50.431879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.194:53689)
0929 10:53:50.432050 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:50.432096 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":60,"threads_started":1}
I20260929 10:53:50.556255 5417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:50.556052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.195:41805)
0929 10:53:50.556165 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:50.556202 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":57,"threads_started":1}
I20260929 10:53:51.356931 5418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:51.356672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.193:60273)
0929 10:53:51.356850 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:51.356882 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":118,"threads_started":1}
I20260929 10:53:51.432958 5419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:51.432749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.194:35439)
0929 10:53:51.432874 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:51.432914 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":55,"threads_started":1}
I20260929 10:53:51.556135 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 4928
I20260929 10:53:51.558184 5420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:51.557982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:39195 (local address 127.20.80.195:60155)
0929 10:53:51.558110 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:51.558142 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.254:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":76,"threads_started":1}
I20260929 10:53:51.560930 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 5098
I20260929 10:53:51.565379 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 5233
2026-09-29T10:53:51Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58293 ms)
[----------] 30 tests from SecurityITest (235446 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:53:51 dist-test-slave-vqch krb5kdc[5423](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:53:51 dist-test-slave-vqch krb5kdc[5423](info): set up 2 sockets
Sep 29 10:53:51 dist-test-slave-vqch krb5kdc[5423](info): commencing operation
krb5kdc: starting...
W20260929 10:53:52.612522 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:53:52 dist-test-slave-vqch krb5kdc[5423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:53:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:53:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.80.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
I20260929 10:53:52.735867 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:38399
--webserver_interface=127.20.80.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:45101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:38399
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:53:52.815753 5436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:53:52.815959 5436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:53:52.815982 5436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:53:52.817081 5436 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:53:52.817117 5436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:53:52.817131 5436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:53:52.817143 5436 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:53:52.817157 5436 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:53:52.818555 5436 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:45101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:38399
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:38399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:53:52.818745 5436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:53:52.818912 5436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:53:52.823709 5444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:52.823709 5441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:52.823761 5442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:53:52.824033 5436 server_base.cc:1034] running on GCE node
I20260929 10:53:52.824307 5436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:53:52.824591 5436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:53:52.825752 5436 hybrid_clock.cc:648] HybridClock initialized: now 1790679232825705 us; error 92 us; skew 500 ppm
Sep 29 10:53:52 dist-test-slave-vqch krb5kdc[5423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679232, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:53:52.828222 5436 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:53:52.828678 5436 webserver.cc:460] Webserver started at http://127.20.80.254:43241/ using document root <none> and password file <none>
I20260929 10:53:52.828862 5436 fs_manager.cc:359] Metadata directory not provided
I20260929 10:53:52.828902 5436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:53:52.829047 5436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:53:52.830080 5436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "dd7fa7bed23748beb09c2dbc76c17772"
format_stamp: "Formatted at 2026-09-29 10:53:52 on dist-test-slave-vqch"
server_key: "c81099eb81767f5ee9926f27a32a43b7"
server_key_iv: "706317d8f2e45f78c4256c03303ce40e"
server_key_version: "encryptionkey@0"
I20260929 10:53:52.830385 5436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "dd7fa7bed23748beb09c2dbc76c17772"
format_stamp: "Formatted at 2026-09-29 10:53:52 on dist-test-slave-vqch"
server_key: "c81099eb81767f5ee9926f27a32a43b7"
server_key_iv: "706317d8f2e45f78c4256c03303ce40e"
server_key_version: "encryptionkey@0"
I20260929 10:53:52.831677 5436 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 10:53:52.832403 5451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:52.832593 5436 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:53:52.832669 5436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "dd7fa7bed23748beb09c2dbc76c17772"
format_stamp: "Formatted at 2026-09-29 10:53:52 on dist-test-slave-vqch"
server_key: "c81099eb81767f5ee9926f27a32a43b7"
server_key_iv: "706317d8f2e45f78c4256c03303ce40e"
server_key_version: "encryptionkey@0"
I20260929 10:53:52.832724 5436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:53:52.849304 5436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:53:52.851437 5436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:53:52.851557 5436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:53:52.855945 5436 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:38399
I20260929 10:53:52.856016 5504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:38399 every 8 connection(s)
I20260929 10:53:52.856387 5436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:53:52.857395 5505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:53:52.859784 5505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd7fa7bed23748beb09c2dbc76c17772: Bootstrap starting.
I20260929 10:53:52.860389 5505 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd7fa7bed23748beb09c2dbc76c17772: Neither blocks nor log segments found. Creating new log.
I20260929 10:53:52.860703 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 5436
I20260929 10:53:52.860823 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:53:52.860798 5505 log.cc:824] T 00000000000000000000000000000000 P dd7fa7bed23748beb09c2dbc76c17772: Log is configured to *not* fsync() on all Append() calls
I20260929 10:53:52.861092 20803 external_mini_cluster.cc:1428] Setting key e23ab3c1ab5c5574c3b8450d8900699d
I20260929 10:53:52.861692 5505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd7fa7bed23748beb09c2dbc76c17772: No bootstrap required, opened a new log
I20260929 10:53:52.863441 5505 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd7fa7bed23748beb09c2dbc76c17772 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd7fa7bed23748beb09c2dbc76c17772" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 38399 } }
I20260929 10:53:52.863586 5505 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd7fa7bed23748beb09c2dbc76c17772 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:53:52.863610 5505 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd7fa7bed23748beb09c2dbc76c17772 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7fa7bed23748beb09c2dbc76c17772, State: Initialized, Role: FOLLOWER
I20260929 10:53:52.863822 5505 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd7fa7bed23748beb09c2dbc76c17772 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd7fa7bed23748beb09c2dbc76c17772" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 38399 } }
I20260929 10:53:52.863925 5505 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd7fa7bed23748beb09c2dbc76c17772 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:53:52.863963 5505 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd7fa7bed23748beb09c2dbc76c17772 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:53:52.864017 5505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd7fa7bed23748beb09c2dbc76c17772 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:53:52.865159 5505 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd7fa7bed23748beb09c2dbc76c17772 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd7fa7bed23748beb09c2dbc76c17772" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 38399 } }
I20260929 10:53:52.865259 5505 leader_election.cc:302] T 00000000000000000000000000000000 P dd7fa7bed23748beb09c2dbc76c17772 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd7fa7bed23748beb09c2dbc76c17772; no voters:
I20260929 10:53:52.865453 5505 leader_election.cc:288] T 00000000000000000000000000000000 P dd7fa7bed23748beb09c2dbc76c17772 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:53:52.865500 5510 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd7fa7bed23748beb09c2dbc76c17772 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:53:52.865688 5510 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd7fa7bed23748beb09c2dbc76c17772 [term 1 LEADER]: Becoming Leader. State: Replica: dd7fa7bed23748beb09c2dbc76c17772, State: Running, Role: LEADER
I20260929 10:53:52.865717 5505 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd7fa7bed23748beb09c2dbc76c17772 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:53:52.865831 5510 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd7fa7bed23748beb09c2dbc76c17772 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd7fa7bed23748beb09c2dbc76c17772" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 38399 } }
I20260929 10:53:52.866366 5511 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd7fa7bed23748beb09c2dbc76c17772 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd7fa7bed23748beb09c2dbc76c17772" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd7fa7bed23748beb09c2dbc76c17772" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 38399 } } }
I20260929 10:53:52.866446 5511 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd7fa7bed23748beb09c2dbc76c17772 [sys.catalog]: This master's current role is: LEADER
I20260929 10:53:52.866708 5512 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd7fa7bed23748beb09c2dbc76c17772 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd7fa7bed23748beb09c2dbc76c17772. Latest consensus state: current_term: 1 leader_uuid: "dd7fa7bed23748beb09c2dbc76c17772" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd7fa7bed23748beb09c2dbc76c17772" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 38399 } } }
I20260929 10:53:52.866832 5512 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd7fa7bed23748beb09c2dbc76c17772 [sys.catalog]: This master's current role is: LEADER
I20260929 10:53:52.867197 5519 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:53:52.867713 5519 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:53:52.869529 5519 catalog_manager.cc:1347] Generated new cluster ID: d5e0a3f0951e4494a2f80868dae9e83e
I20260929 10:53:52.869583 5519 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 10:53:52 dist-test-slave-vqch krb5kdc[5423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:53:52.874701 5509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:52.861701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45836 (local address 127.20.80.254:38399)
0929 10:53:52.861895 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:52.861905 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:52.861945 (+ 40us) server_negotiation.cc:407] Connection header received
0929 10:53:52.863430 (+ 1485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:52.863439 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:52.863500 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:52.863699 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:53:52.864573 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:52.865955 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:52.867167 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:52.867845 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:52.867956 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:52.870872 (+ 2916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:53:52.870876 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:53:52.870877 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:53:52.870889 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:53:52.872935 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:52.873347 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:52.873351 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:52.873351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:52.873493 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:52.873714 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:52.873715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:52.873715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:52.874103 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:53:52.874263 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:52.874343 (+ 80us) server_negotiation.cc:299] Negotiation successful
0929 10:53:52.874493 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":88,"threads_started":1}
I20260929 10:53:52.883571 5519 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:53:52.884281 5519 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:53:52.899366 5519 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd7fa7bed23748beb09c2dbc76c17772: Generated new TSK 0
I20260929 10:53:52.899611 5519 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.80.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
I20260929 10:53:52.940480 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.193:0
--local_ip_for_outbound_sockets=127.20.80.193
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:38399
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:45101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:53:53.021749 5531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:53:53.021973 5531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:53:53.022013 5531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:53:53.023182 5531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:53:53.023249 5531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:53:53.023293 5531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.193
I20260929 10:53:53.024749 5531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:45101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:38399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.193
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:53:53.025071 5531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:53:53.025306 5531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:53:53.030174 5536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:53.030174 5539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:53.030309 5537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:53:53.030639 5531 server_base.cc:1034] running on GCE node
I20260929 10:53:53.030850 5531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:53:53.031090 5531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:53:53.032253 5531 hybrid_clock.cc:648] HybridClock initialized: now 1790679233032241 us; error 41 us; skew 500 ppm
Sep 29 10:53:53 dist-test-slave-vqch krb5kdc[5423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679233, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:53:53.035102 5531 init.cc:377] Logged in from keytab as kudu/127.20.80.193@KRBTEST.COM (short username kudu)
I20260929 10:53:53.035575 5531 webserver.cc:460] Webserver started at http://127.20.80.193:32779/ using document root <none> and password file <none>
I20260929 10:53:53.035805 5531 fs_manager.cc:359] Metadata directory not provided
I20260929 10:53:53.035851 5531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:53:53.036010 5531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:53:53.037211 5531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-0/data/instance:
uuid: "72cefb274fb446f5a898cc40ba694850"
format_stamp: "Formatted at 2026-09-29 10:53:53 on dist-test-slave-vqch"
server_key: "afdead0b4c3d3eb0a7de4d5eab656ba8"
server_key_iv: "8733d752379fd8876071ee2f14a29d4b"
server_key_version: "encryptionkey@0"
I20260929 10:53:53.037566 5531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance:
uuid: "72cefb274fb446f5a898cc40ba694850"
format_stamp: "Formatted at 2026-09-29 10:53:53 on dist-test-slave-vqch"
server_key: "afdead0b4c3d3eb0a7de4d5eab656ba8"
server_key_iv: "8733d752379fd8876071ee2f14a29d4b"
server_key_version: "encryptionkey@0"
I20260929 10:53:53.038899 5531 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260929 10:53:53.039690 5546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:53.039902 5531 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 10:53:53.039994 5531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-0/wal
uuid: "72cefb274fb446f5a898cc40ba694850"
format_stamp: "Formatted at 2026-09-29 10:53:53 on dist-test-slave-vqch"
server_key: "afdead0b4c3d3eb0a7de4d5eab656ba8"
server_key_iv: "8733d752379fd8876071ee2f14a29d4b"
server_key_version: "encryptionkey@0"
I20260929 10:53:53.040067 5531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:53:53.047961 5531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:53:53.050057 5531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:53:53.050213 5531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:53:53.050432 5531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:53:53.050726 5531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:53:53.050758 5531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:53.050798 5531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:53:53.050824 5531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:53.056447 5531 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.193:41553
I20260929 10:53:53.056522 5660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.193:41553 every 8 connection(s)
I20260929 10:53:53.056828 5531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
Sep 29 10:53:53 dist-test-slave-vqch krb5kdc[5423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679233, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:53:53.065681 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 5531
I20260929 10:53:53.065804 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance
I20260929 10:53:53.066064 20803 external_mini_cluster.cc:1428] Setting key 85f487216617149a8df46774814f4182
I20260929 10:53:53.066069 5509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:53.057541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:51855 (local address 127.20.80.254:38399)
0929 10:53:53.057602 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:53.057604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:53.057800 (+ 196us) server_negotiation.cc:407] Connection header received
0929 10:53:53.058324 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:53.058325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:53.058341 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:53.058393 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:53:53.059324 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.059828 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.060814 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.061108 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.061196 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:53.063213 (+ 2017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:53:53.063214 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:53:53.063215 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:53:53.063222 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:53:53.064696 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:53.065121 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:53.065121 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:53.065122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:53.065172 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:53.065379 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:53.065379 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:53.065379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:53.065479 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:53:53.065528 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:53.065912 (+ 384us) server_negotiation.cc:299] Negotiation successful
0929 10:53:53.066002 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 10:53:53.066252 5663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:53.057519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:38399 (local address 127.20.80.193:51855)
0929 10:53:53.057675 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:53.057722 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:53.058218 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:53.058460 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:53.058464 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:53.059031 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:53:53.059216 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:53.059220 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.059930 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.059931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:53.060713 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:53.060715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.061201 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.061202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:53.061328 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:53.061774 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:53:53.061781 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:53:53.063126 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:53:53.064781 (+ 1655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:53:53.064787 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:53:53.064789 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:53:53.065050 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:53:53.065232 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:53:53.065232 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:53:53.065232 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:53:53.065326 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:53:53.065540 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:53:53.065547 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:53:53.065709 (+ 162us) client_negotiation.cc:769] Sending connection context
0929 10:53:53.065835 (+ 126us) client_negotiation.cc:240] Negotiation successful
0929 10:53:53.066013 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":76,"threads_started":1}
I20260929 10:53:53.066871 5661 heartbeater.cc:347] Connected to a master server at 127.20.80.254:38399
I20260929 10:53:53.067006 5661 heartbeater.cc:464] Registering TS with master...
I20260929 10:53:53.067176 5661 heartbeater.cc:511] Master 127.20.80.254:38399 requested a full tablet report, sending...
I20260929 10:53:53.067730 5468 ts_manager.cc:194] Registered new tserver with Master: 72cefb274fb446f5a898cc40ba694850 (127.20.80.193:41553)
I20260929 10:53:53.068336 5468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.193@KRBTEST.COM'} at 127.20.80.193:51855
WARNING: no policy specified for kudu/127.20.80.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
I20260929 10:53:53.091988 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.194:0
--local_ip_for_outbound_sockets=127.20.80.194
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:38399
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:45101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:53:53.172785 5666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:53:53.173002 5666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:53:53.173071 5666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:53:53.174163 5666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:53:53.174197 5666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:53:53.174222 5666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.194
I20260929 10:53:53.175585 5666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:45101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:38399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.194
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:53:53.175773 5666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:53:53.175945 5666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:53:53.181192 5672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:53:53.181406 5666 server_base.cc:1034] running on GCE node
W20260929 10:53:53.181170 5674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:53.181157 5671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:53:53.181734 5666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:53:53.181986 5666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:53:53.183156 5666 hybrid_clock.cc:648] HybridClock initialized: now 1790679233183159 us; error 44 us; skew 500 ppm
Sep 29 10:53:53 dist-test-slave-vqch krb5kdc[5423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679233, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:53:53.185657 5666 init.cc:377] Logged in from keytab as kudu/127.20.80.194@KRBTEST.COM (short username kudu)
I20260929 10:53:53.186151 5666 webserver.cc:460] Webserver started at http://127.20.80.194:40407/ using document root <none> and password file <none>
I20260929 10:53:53.186407 5666 fs_manager.cc:359] Metadata directory not provided
I20260929 10:53:53.186475 5666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:53:53.186614 5666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:53:53.187673 5666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-1/data/instance:
uuid: "81be471d87be456caec2a5384f121ac2"
format_stamp: "Formatted at 2026-09-29 10:53:53 on dist-test-slave-vqch"
server_key: "cee6cbb71b2dc4bf8e81deceaeb7c7b2"
server_key_iv: "387fb3607be96fbd00a6ba1c094acd1a"
server_key_version: "encryptionkey@0"
I20260929 10:53:53.188052 5666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance:
uuid: "81be471d87be456caec2a5384f121ac2"
format_stamp: "Formatted at 2026-09-29 10:53:53 on dist-test-slave-vqch"
server_key: "cee6cbb71b2dc4bf8e81deceaeb7c7b2"
server_key_iv: "387fb3607be96fbd00a6ba1c094acd1a"
server_key_version: "encryptionkey@0"
I20260929 10:53:53.189474 5666 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 10:53:53.190299 5681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:53.190615 5666 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:53:53.190721 5666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-1/wal
uuid: "81be471d87be456caec2a5384f121ac2"
format_stamp: "Formatted at 2026-09-29 10:53:53 on dist-test-slave-vqch"
server_key: "cee6cbb71b2dc4bf8e81deceaeb7c7b2"
server_key_iv: "387fb3607be96fbd00a6ba1c094acd1a"
server_key_version: "encryptionkey@0"
I20260929 10:53:53.190817 5666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:53:53.203071 5666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:53:53.205267 5666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:53:53.205437 5666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:53:53.205681 5666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:53:53.205991 5666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:53:53.206043 5666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:53.206082 5666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:53:53.206106 5666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:53.211905 5666 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.194:38877
I20260929 10:53:53.211954 5795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.194:38877 every 8 connection(s)
I20260929 10:53:53.212270 5666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
I20260929 10:53:53.216874 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 5666
I20260929 10:53:53.217042 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance
I20260929 10:53:53.217279 20803 external_mini_cluster.cc:1428] Setting key e4cce19d3107ee95a4abf4e4849ded98
Sep 29 10:53:53 dist-test-slave-vqch krb5kdc[5423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679233, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:53:53.222903 5509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:53.213006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:38783 (local address 127.20.80.254:38399)
0929 10:53:53.213071 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:53.213076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:53.213302 (+ 226us) server_negotiation.cc:407] Connection header received
0929 10:53:53.213796 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:53.213797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:53.213808 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:53.213867 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:53:53.214804 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.215351 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.216360 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.216559 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.216627 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:53.219245 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:53:53.219247 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:53:53.219247 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:53:53.219256 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:53:53.221211 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:53.221723 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:53.221724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:53.221724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:53.221783 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:53.222105 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:53.222106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:53.222107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:53.222268 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:53:53.222327 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:53.222740 (+ 413us) server_negotiation.cc:299] Negotiation successful
0929 10:53:53.222826 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 10:53:53.223146 5798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:53.212966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:38399 (local address 127.20.80.194:38783)
0929 10:53:53.213166 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:53.213208 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:53.213696 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:53.213923 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:53.213926 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:53.214504 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:53:53.214702 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:53.214707 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.215443 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.215444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:53.216250 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:53.216253 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.216720 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.216721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:53.216834 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:53.217538 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:53:53.217548 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:53:53.219141 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:53:53.221294 (+ 2153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:53:53.221302 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:53:53.221305 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:53:53.221629 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:53:53.221857 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:53:53.221858 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:53:53.221859 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:53:53.222012 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:53:53.222343 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:53:53.222350 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:53:53.222533 (+ 183us) client_negotiation.cc:769] Sending connection context
0929 10:53:53.222664 (+ 131us) client_negotiation.cc:240] Negotiation successful
0929 10:53:53.222858 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":121,"threads_started":1}
I20260929 10:53:53.223742 5796 heartbeater.cc:347] Connected to a master server at 127.20.80.254:38399
I20260929 10:53:53.223840 5796 heartbeater.cc:464] Registering TS with master...
I20260929 10:53:53.224047 5796 heartbeater.cc:511] Master 127.20.80.254:38399 requested a full tablet report, sending...
I20260929 10:53:53.224470 5468 ts_manager.cc:194] Registered new tserver with Master: 81be471d87be456caec2a5384f121ac2 (127.20.80.194:38877)
I20260929 10:53:53.224853 5468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.194@KRBTEST.COM'} at 127.20.80.194:38783
WARNING: no policy specified for kudu/127.20.80.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
I20260929 10:53:53.248251 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.195:0
--local_ip_for_outbound_sockets=127.20.80.195
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:38399
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:45101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:53:53.330276 5801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:53:53.330526 5801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:53:53.330564 5801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:53:53.331705 5801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:53:53.331784 5801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:53:53.331830 5801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.195
I20260929 10:53:53.333469 5801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:45101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:38399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.195
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:53:53.333745 5801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:53:53.333981 5801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:53:53.339008 5806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:53.339008 5807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:53.339038 5809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:53:53.339517 5801 server_base.cc:1034] running on GCE node
I20260929 10:53:53.339709 5801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:53:53.339936 5801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:53:53.341053 5801 hybrid_clock.cc:648] HybridClock initialized: now 1790679233341037 us; error 27 us; skew 500 ppm
Sep 29 10:53:53 dist-test-slave-vqch krb5kdc[5423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679233, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:53:53.343654 5801 init.cc:377] Logged in from keytab as kudu/127.20.80.195@KRBTEST.COM (short username kudu)
I20260929 10:53:53.344179 5801 webserver.cc:460] Webserver started at http://127.20.80.195:40347/ using document root <none> and password file <none>
I20260929 10:53:53.344431 5801 fs_manager.cc:359] Metadata directory not provided
I20260929 10:53:53.344499 5801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:53:53.344632 5801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:53:53.345727 5801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-2/data/instance:
uuid: "c14cba3431e240429e15842b4736a423"
format_stamp: "Formatted at 2026-09-29 10:53:53 on dist-test-slave-vqch"
server_key: "eb36697333138f65bfa4d3a5f5889e70"
server_key_iv: "d3b0cba9a228008b458b39adcb0969ac"
server_key_version: "encryptionkey@0"
I20260929 10:53:53.346103 5801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance:
uuid: "c14cba3431e240429e15842b4736a423"
format_stamp: "Formatted at 2026-09-29 10:53:53 on dist-test-slave-vqch"
server_key: "eb36697333138f65bfa4d3a5f5889e70"
server_key_iv: "d3b0cba9a228008b458b39adcb0969ac"
server_key_version: "encryptionkey@0"
I20260929 10:53:53.347436 5801 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 10:53:53.348287 5816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:53.348531 5801 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:53:53.348632 5801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-2/wal
uuid: "c14cba3431e240429e15842b4736a423"
format_stamp: "Formatted at 2026-09-29 10:53:53 on dist-test-slave-vqch"
server_key: "eb36697333138f65bfa4d3a5f5889e70"
server_key_iv: "d3b0cba9a228008b458b39adcb0969ac"
server_key_version: "encryptionkey@0"
I20260929 10:53:53.348718 5801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:53:53.365845 5801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:53:53.368579 5801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:53:53.368747 5801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:53:53.369072 5801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:53:53.369406 5801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:53:53.369462 5801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:53.369504 5801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:53:53.369531 5801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:53.375528 5801 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.195:38047
I20260929 10:53:53.375571 5929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.195:38047 every 8 connection(s)
I20260929 10:53:53.375896 5801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
Sep 29 10:53:53 dist-test-slave-vqch krb5kdc[5423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679233, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:53:53.383749 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 5801
I20260929 10:53:53.383870 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance
I20260929 10:53:53.384075 20803 external_mini_cluster.cc:1428] Setting key c11c43591939a54f958ef98fdfa2b45a
I20260929 10:53:53.385588 5509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:53.376643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:41263 (local address 127.20.80.254:38399)
0929 10:53:53.376698 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:53.376700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:53.376921 (+ 221us) server_negotiation.cc:407] Connection header received
0929 10:53:53.377506 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:53.377508 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:53.377521 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:53.377577 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:53:53.378526 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.379040 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.380046 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.380243 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.380327 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:53.382666 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:53:53.382667 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:53:53.382668 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:53:53.382676 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:53:53.384034 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:53.384549 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:53.384550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:53.384550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:53.384599 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:53.384874 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:53.384875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:53.384875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:53.385052 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:53:53.385136 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:53.385414 (+ 278us) server_negotiation.cc:299] Negotiation successful
0929 10:53:53.385495 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260929 10:53:53.385795 5932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:53.376601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:38399 (local address 127.20.80.195:41263)
0929 10:53:53.376792 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:53.376829 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:53.377380 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:53.377657 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:53.377662 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:53.378224 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:53:53.378426 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:53.378431 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.379136 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.379137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:53.379941 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:53.379943 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.380337 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.380338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:53.380475 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:53.381047 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:53:53.381056 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:53:53.382568 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:53:53.384121 (+ 1553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:53:53.384128 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:53:53.384131 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:53:53.384422 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:53:53.384672 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:53:53.384673 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:53:53.384673 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:53:53.384797 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:53:53.385126 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:53:53.385133 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:53:53.385280 (+ 147us) client_negotiation.cc:769] Sending connection context
0929 10:53:53.385393 (+ 113us) client_negotiation.cc:240] Negotiation successful
0929 10:53:53.385559 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":101,"threads_started":1}
I20260929 10:53:53.386308 5930 heartbeater.cc:347] Connected to a master server at 127.20.80.254:38399
I20260929 10:53:53.386430 5930 heartbeater.cc:464] Registering TS with master...
I20260929 10:53:53.386596 5930 heartbeater.cc:511] Master 127.20.80.254:38399 requested a full tablet report, sending...
I20260929 10:53:53.386924 5468 ts_manager.cc:194] Registered new tserver with Master: c14cba3431e240429e15842b4736a423 (127.20.80.195:38047)
I20260929 10:53:53.387317 5468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.195@KRBTEST.COM'} at 127.20.80.195:41263
I20260929 10:53:53.396049 20803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:53:53.403908 5509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:53.397409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43848 (local address 127.20.80.254:38399)
0929 10:53:53.397485 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:53.397488 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:53.397589 (+ 101us) server_negotiation.cc:407] Connection header received
0929 10:53:53.397717 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:53.397718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:53.397728 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:53.397794 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:53:53.398386 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.398885 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.399686 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.399882 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.399979 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:53.401313 (+ 1334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:53:53.401315 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:53:53.401315 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:53:53.401324 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:53:53.402555 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:53.402942 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:53.402943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:53.402944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:53.403011 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:53.403248 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:53.403249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:53.403250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:53.403415 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:53:53.403535 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:53.403682 (+ 147us) server_negotiation.cc:299] Negotiation successful
0929 10:53:53.403783 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 10:53:53.405535 5468 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43848:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:53:53.407171 5468 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:53:53.414187 5944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:53.409682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:41553 (local address 127.0.0.1:46320)
0929 10:53:53.410246 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:53.410256 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:53.410314 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:53.410702 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:53.410703 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:53.410706 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:53:53.410755 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:53.410757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.411948 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.411948 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:53.412850 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:53.412852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.413854 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.413855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:53.413948 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:53.413965 (+ 17us) client_negotiation.cc:769] Sending connection context
0929 10:53:53.414046 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 10:53:53.414110 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":89,"threads_started":1}
I20260929 10:53:53.414254 5943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:53.409602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54450 (local address 127.20.80.194:38877)
0929 10:53:53.409858 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:53.409862 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:53.409871 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:53:53.409907 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:53.409909 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:53.409922 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:53.410029 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:53:53.410389 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.411341 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.413108 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.413775 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.413838 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:53.413886 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:53.414102 (+ 216us) server_negotiation.cc:299] Negotiation successful
0929 10:53:53.414164 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":127,"threads_started":1}
I20260929 10:53:53.414428 5946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:53.410058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46320 (local address 127.20.80.193:41553)
0929 10:53:53.410219 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:53.410224 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:53.410534 (+ 310us) server_negotiation.cc:407] Connection header received
0929 10:53:53.410583 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:53.410585 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:53.410600 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:53.410682 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:53:53.410822 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.411850 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.412937 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.413725 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.413811 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:53.413876 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:53.414307 (+ 431us) server_negotiation.cc:299] Negotiation successful
0929 10:53:53.414358 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":96,"threads_started":1}
I20260929 10:53:53.414701 5942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:53.409453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:38877 (local address 127.0.0.1:54450)
0929 10:53:53.409641 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:53.409683 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:53.409746 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:53.410035 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:53.410039 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:53.410042 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:53:53.410134 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:53.410137 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.411458 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.411459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:53.413021 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:53.413024 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.413855 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.413856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:53.413956 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:53.413965 (+ 9us) client_negotiation.cc:769] Sending connection context
0929 10:53:53.414046 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 10:53:53.414591 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":522,"thread_start_us":75,"threads_started":1}
I20260929 10:53:53.415143 5594 tablet_service.cc:1467] Processing CreateTablet for tablet aa096bb1e72f44559605f6d78e19b339 (DEFAULT_TABLE table=test-table [id=794cd58b28134b5c84432d17a14ac46a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:53:53.415362 5941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:53.409489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:38047 (local address 127.0.0.1:55354)
0929 10:53:53.410128 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:53.410140 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:53.410194 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:53.410431 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:53.410432 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:53.410434 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:53:53.410490 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:53.410491 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.411789 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.411790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:53.413039 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:53.413040 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.415110 (+ 2070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.415111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:53.415202 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:53.415206 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:53:53.415254 (+ 48us) client_negotiation.cc:240] Negotiation successful
0929 10:53:53.415295 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":90,"threads_started":1}
I20260929 10:53:53.415566 5594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa096bb1e72f44559605f6d78e19b339. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:53:53.415653 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:53.409844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55354 (local address 127.20.80.195:38047)
0929 10:53:53.410015 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:53.410020 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:53.410265 (+ 245us) server_negotiation.cc:407] Connection header received
0929 10:53:53.410319 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:53.410323 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:53.410340 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:53.410425 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:53:53.410577 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.411676 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.413921 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.414935 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.415004 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:53.415069 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:53.415529 (+ 460us) server_negotiation.cc:299] Negotiation successful
0929 10:53:53.415573 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":103,"threads_started":1}
I20260929 10:53:53.415688 5729 tablet_service.cc:1467] Processing CreateTablet for tablet aa096bb1e72f44559605f6d78e19b339 (DEFAULT_TABLE table=test-table [id=794cd58b28134b5c84432d17a14ac46a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:53:53.416034 5729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa096bb1e72f44559605f6d78e19b339. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:53:53.416265 5864 tablet_service.cc:1467] Processing CreateTablet for tablet aa096bb1e72f44559605f6d78e19b339 (DEFAULT_TABLE table=test-table [id=794cd58b28134b5c84432d17a14ac46a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:53:53.416580 5864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa096bb1e72f44559605f6d78e19b339. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:53:53.419003 5947 tablet_bootstrap.cc:492] T aa096bb1e72f44559605f6d78e19b339 P 81be471d87be456caec2a5384f121ac2: Bootstrap starting.
I20260929 10:53:53.419701 5947 tablet_bootstrap.cc:654] T aa096bb1e72f44559605f6d78e19b339 P 81be471d87be456caec2a5384f121ac2: Neither blocks nor log segments found. Creating new log.
I20260929 10:53:53.419919 5948 tablet_bootstrap.cc:492] T aa096bb1e72f44559605f6d78e19b339 P 72cefb274fb446f5a898cc40ba694850: Bootstrap starting.
I20260929 10:53:53.420374 5947 log.cc:824] T aa096bb1e72f44559605f6d78e19b339 P 81be471d87be456caec2a5384f121ac2: Log is configured to *not* fsync() on all Append() calls
I20260929 10:53:53.420600 5948 tablet_bootstrap.cc:654] T aa096bb1e72f44559605f6d78e19b339 P 72cefb274fb446f5a898cc40ba694850: Neither blocks nor log segments found. Creating new log.
I20260929 10:53:53.420766 5949 tablet_bootstrap.cc:492] T aa096bb1e72f44559605f6d78e19b339 P c14cba3431e240429e15842b4736a423: Bootstrap starting.
I20260929 10:53:53.421226 5947 tablet_bootstrap.cc:492] T aa096bb1e72f44559605f6d78e19b339 P 81be471d87be456caec2a5384f121ac2: No bootstrap required, opened a new log
I20260929 10:53:53.421242 5948 log.cc:824] T aa096bb1e72f44559605f6d78e19b339 P 72cefb274fb446f5a898cc40ba694850: Log is configured to *not* fsync() on all Append() calls
I20260929 10:53:53.421314 5947 ts_tablet_manager.cc:1397] T aa096bb1e72f44559605f6d78e19b339 P 81be471d87be456caec2a5384f121ac2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:53:53.421528 5949 tablet_bootstrap.cc:654] T aa096bb1e72f44559605f6d78e19b339 P c14cba3431e240429e15842b4736a423: Neither blocks nor log segments found. Creating new log.
I20260929 10:53:53.422030 5948 tablet_bootstrap.cc:492] T aa096bb1e72f44559605f6d78e19b339 P 72cefb274fb446f5a898cc40ba694850: No bootstrap required, opened a new log
I20260929 10:53:53.422078 5949 log.cc:824] T aa096bb1e72f44559605f6d78e19b339 P c14cba3431e240429e15842b4736a423: Log is configured to *not* fsync() on all Append() calls
I20260929 10:53:53.422106 5948 ts_tablet_manager.cc:1397] T aa096bb1e72f44559605f6d78e19b339 P 72cefb274fb446f5a898cc40ba694850: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:53:53.423200 5949 tablet_bootstrap.cc:492] T aa096bb1e72f44559605f6d78e19b339 P c14cba3431e240429e15842b4736a423: No bootstrap required, opened a new log
I20260929 10:53:53.423270 5949 ts_tablet_manager.cc:1397] T aa096bb1e72f44559605f6d78e19b339 P c14cba3431e240429e15842b4736a423: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 10:53:53.423213 5947 raft_consensus.cc:348] T aa096bb1e72f44559605f6d78e19b339 P 81be471d87be456caec2a5384f121ac2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81be471d87be456caec2a5384f121ac2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 38877 } } peers { permanent_uuid: "c14cba3431e240429e15842b4736a423" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 38047 } } peers { permanent_uuid: "72cefb274fb446f5a898cc40ba694850" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 41553 } }
I20260929 10:53:53.423434 5947 raft_consensus.cc:374] T aa096bb1e72f44559605f6d78e19b339 P 81be471d87be456caec2a5384f121ac2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:53:53.423478 5947 raft_consensus.cc:729] T aa096bb1e72f44559605f6d78e19b339 P 81be471d87be456caec2a5384f121ac2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81be471d87be456caec2a5384f121ac2, State: Initialized, Role: FOLLOWER
I20260929 10:53:53.423622 5947 consensus_queue.cc:260] T aa096bb1e72f44559605f6d78e19b339 P 81be471d87be456caec2a5384f121ac2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81be471d87be456caec2a5384f121ac2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 38877 } } peers { permanent_uuid: "c14cba3431e240429e15842b4736a423" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 38047 } } peers { permanent_uuid: "72cefb274fb446f5a898cc40ba694850" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 41553 } }
I20260929 10:53:53.423908 5947 ts_tablet_manager.cc:1428] T aa096bb1e72f44559605f6d78e19b339 P 81be471d87be456caec2a5384f121ac2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 10:53:53.423911 5796 heartbeater.cc:503] Master 127.20.80.254:38399 was elected leader, sending a full tablet report...
I20260929 10:53:53.423928 5948 raft_consensus.cc:348] T aa096bb1e72f44559605f6d78e19b339 P 72cefb274fb446f5a898cc40ba694850 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81be471d87be456caec2a5384f121ac2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 38877 } } peers { permanent_uuid: "c14cba3431e240429e15842b4736a423" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 38047 } } peers { permanent_uuid: "72cefb274fb446f5a898cc40ba694850" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 41553 } }
I20260929 10:53:53.424088 5948 raft_consensus.cc:374] T aa096bb1e72f44559605f6d78e19b339 P 72cefb274fb446f5a898cc40ba694850 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:53:53.424122 5948 raft_consensus.cc:729] T aa096bb1e72f44559605f6d78e19b339 P 72cefb274fb446f5a898cc40ba694850 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72cefb274fb446f5a898cc40ba694850, State: Initialized, Role: FOLLOWER
I20260929 10:53:53.424269 5948 consensus_queue.cc:260] T aa096bb1e72f44559605f6d78e19b339 P 72cefb274fb446f5a898cc40ba694850 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81be471d87be456caec2a5384f121ac2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 38877 } } peers { permanent_uuid: "c14cba3431e240429e15842b4736a423" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 38047 } } peers { permanent_uuid: "72cefb274fb446f5a898cc40ba694850" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 41553 } }
I20260929 10:53:53.424542 5948 ts_tablet_manager.cc:1428] T aa096bb1e72f44559605f6d78e19b339 P 72cefb274fb446f5a898cc40ba694850: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 10:53:53.424552 5661 heartbeater.cc:503] Master 127.20.80.254:38399 was elected leader, sending a full tablet report...
I20260929 10:53:53.425375 5949 raft_consensus.cc:348] T aa096bb1e72f44559605f6d78e19b339 P c14cba3431e240429e15842b4736a423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81be471d87be456caec2a5384f121ac2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 38877 } } peers { permanent_uuid: "c14cba3431e240429e15842b4736a423" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 38047 } } peers { permanent_uuid: "72cefb274fb446f5a898cc40ba694850" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 41553 } }
I20260929 10:53:53.425523 5949 raft_consensus.cc:374] T aa096bb1e72f44559605f6d78e19b339 P c14cba3431e240429e15842b4736a423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:53:53.425563 5949 raft_consensus.cc:729] T aa096bb1e72f44559605f6d78e19b339 P c14cba3431e240429e15842b4736a423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c14cba3431e240429e15842b4736a423, State: Initialized, Role: FOLLOWER
I20260929 10:53:53.425683 5949 consensus_queue.cc:260] T aa096bb1e72f44559605f6d78e19b339 P c14cba3431e240429e15842b4736a423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81be471d87be456caec2a5384f121ac2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 38877 } } peers { permanent_uuid: "c14cba3431e240429e15842b4736a423" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 38047 } } peers { permanent_uuid: "72cefb274fb446f5a898cc40ba694850" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 41553 } }
I20260929 10:53:53.425961 5949 ts_tablet_manager.cc:1428] T aa096bb1e72f44559605f6d78e19b339 P c14cba3431e240429e15842b4736a423: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 10:53:53.425989 5930 heartbeater.cc:503] Master 127.20.80.254:38399 was elected leader, sending a full tablet report...
W20260929 10:53:53.462870 5797 tablet.cc:2369] T aa096bb1e72f44559605f6d78e19b339 P 81be471d87be456caec2a5384f121ac2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:53:53.504869 5954 raft_consensus.cc:482] T aa096bb1e72f44559605f6d78e19b339 P 72cefb274fb446f5a898cc40ba694850 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:53:53.505064 5954 raft_consensus.cc:504] T aa096bb1e72f44559605f6d78e19b339 P 72cefb274fb446f5a898cc40ba694850 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81be471d87be456caec2a5384f121ac2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 38877 } } peers { permanent_uuid: "c14cba3431e240429e15842b4736a423" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 38047 } } peers { permanent_uuid: "72cefb274fb446f5a898cc40ba694850" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 41553 } }
I20260929 10:53:53.505517 5954 leader_election.cc:288] T aa096bb1e72f44559605f6d78e19b339 P 72cefb274fb446f5a898cc40ba694850 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 81be471d87be456caec2a5384f121ac2 (127.20.80.194:38877), c14cba3431e240429e15842b4736a423 (127.20.80.195:38047)
I20260929 10:53:53.509722 5663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:53.505726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:38047 (local address 127.20.80.193:59947)
0929 10:53:53.505791 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:53.505803 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:53.505850 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:53.506049 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:53.506050 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:53.506052 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:53:53.506129 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:53.506131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.507228 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.507230 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:53.508531 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:53.508534 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.509416 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.509417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:53.509500 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:53.509502 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 10:53:53.509562 (+ 60us) client_negotiation.cc:240] Negotiation successful
0929 10:53:53.509608 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":46}
I20260929 10:53:53.509745 5943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:53.505834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:48259 (local address 127.20.80.194:38877)
0929 10:53:53.505885 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:53.505888 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:53.506105 (+ 217us) server_negotiation.cc:407] Connection header received
0929 10:53:53.506142 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:53.506143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:53.506154 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:53.506236 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:53:53.506393 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.507162 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.508633 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.509321 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.509409 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:53.509418 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:53.509619 (+ 201us) server_negotiation.cc:299] Negotiation successful
0929 10:53:53.509675 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260929 10:53:53.509769 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:53.505837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:59947 (local address 127.20.80.195:38047)
0929 10:53:53.505896 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:53.505899 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:53.505908 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:53:53.505947 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:53.505948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:53.505955 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:53.506021 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:53:53.506215 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.507124 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.508635 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.509321 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.509408 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:53.509418 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:53.509650 (+ 232us) server_negotiation.cc:299] Negotiation successful
0929 10:53:53.509698 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 10:53:53.509729 5956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:53.505728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:38877 (local address 127.20.80.193:48259)
0929 10:53:53.506028 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:53.506037 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:53.506089 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:53.506249 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:53.506250 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:53.506251 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:53:53.506305 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:53.506306 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.507251 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.507252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:53.508531 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:53.508534 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.509416 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.509417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:53.509496 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:53.509501 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 10:53:53.509562 (+ 61us) client_negotiation.cc:240] Negotiation successful
0929 10:53:53.509607 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":133,"thread_start_us":130,"threads_started":1}
I20260929 10:53:53.510143 5749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa096bb1e72f44559605f6d78e19b339" candidate_uuid: "72cefb274fb446f5a898cc40ba694850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81be471d87be456caec2a5384f121ac2" is_pre_election: true
I20260929 10:53:53.510142 5884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa096bb1e72f44559605f6d78e19b339" candidate_uuid: "72cefb274fb446f5a898cc40ba694850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c14cba3431e240429e15842b4736a423" is_pre_election: true
I20260929 10:53:53.510306 5749 raft_consensus.cc:2454] T aa096bb1e72f44559605f6d78e19b339 P 81be471d87be456caec2a5384f121ac2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72cefb274fb446f5a898cc40ba694850 in term 0.
I20260929 10:53:53.510339 5884 raft_consensus.cc:2454] T aa096bb1e72f44559605f6d78e19b339 P c14cba3431e240429e15842b4736a423 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72cefb274fb446f5a898cc40ba694850 in term 0.
I20260929 10:53:53.510545 5548 leader_election.cc:302] T aa096bb1e72f44559605f6d78e19b339 P 72cefb274fb446f5a898cc40ba694850 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72cefb274fb446f5a898cc40ba694850, 81be471d87be456caec2a5384f121ac2; no voters:
I20260929 10:53:53.510722 5954 raft_consensus.cc:2781] T aa096bb1e72f44559605f6d78e19b339 P 72cefb274fb446f5a898cc40ba694850 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:53:53.510794 5954 raft_consensus.cc:482] T aa096bb1e72f44559605f6d78e19b339 P 72cefb274fb446f5a898cc40ba694850 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:53:53.510814 5954 raft_consensus.cc:3037] T aa096bb1e72f44559605f6d78e19b339 P 72cefb274fb446f5a898cc40ba694850 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:53:53.511619 5954 raft_consensus.cc:504] T aa096bb1e72f44559605f6d78e19b339 P 72cefb274fb446f5a898cc40ba694850 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81be471d87be456caec2a5384f121ac2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 38877 } } peers { permanent_uuid: "c14cba3431e240429e15842b4736a423" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 38047 } } peers { permanent_uuid: "72cefb274fb446f5a898cc40ba694850" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 41553 } }
I20260929 10:53:53.511781 5954 leader_election.cc:288] T aa096bb1e72f44559605f6d78e19b339 P 72cefb274fb446f5a898cc40ba694850 [CANDIDATE]: Term 1 election: Requested vote from peers 81be471d87be456caec2a5384f121ac2 (127.20.80.194:38877), c14cba3431e240429e15842b4736a423 (127.20.80.195:38047)
I20260929 10:53:53.511960 5749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa096bb1e72f44559605f6d78e19b339" candidate_uuid: "72cefb274fb446f5a898cc40ba694850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81be471d87be456caec2a5384f121ac2"
I20260929 10:53:53.511960 5884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa096bb1e72f44559605f6d78e19b339" candidate_uuid: "72cefb274fb446f5a898cc40ba694850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c14cba3431e240429e15842b4736a423"
I20260929 10:53:53.512048 5884 raft_consensus.cc:3037] T aa096bb1e72f44559605f6d78e19b339 P c14cba3431e240429e15842b4736a423 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:53:53.512048 5749 raft_consensus.cc:3037] T aa096bb1e72f44559605f6d78e19b339 P 81be471d87be456caec2a5384f121ac2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:53:53.513046 5884 raft_consensus.cc:2454] T aa096bb1e72f44559605f6d78e19b339 P c14cba3431e240429e15842b4736a423 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72cefb274fb446f5a898cc40ba694850 in term 1.
I20260929 10:53:53.513046 5749 raft_consensus.cc:2454] T aa096bb1e72f44559605f6d78e19b339 P 81be471d87be456caec2a5384f121ac2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72cefb274fb446f5a898cc40ba694850 in term 1.
I20260929 10:53:53.513231 5548 leader_election.cc:302] T aa096bb1e72f44559605f6d78e19b339 P 72cefb274fb446f5a898cc40ba694850 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72cefb274fb446f5a898cc40ba694850, 81be471d87be456caec2a5384f121ac2; no voters:
I20260929 10:53:53.513336 5954 raft_consensus.cc:2781] T aa096bb1e72f44559605f6d78e19b339 P 72cefb274fb446f5a898cc40ba694850 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:53:53.513528 5954 raft_consensus.cc:686] T aa096bb1e72f44559605f6d78e19b339 P 72cefb274fb446f5a898cc40ba694850 [term 1 LEADER]: Becoming Leader. State: Replica: 72cefb274fb446f5a898cc40ba694850, State: Running, Role: LEADER
I20260929 10:53:53.513652 5954 consensus_queue.cc:237] T aa096bb1e72f44559605f6d78e19b339 P 72cefb274fb446f5a898cc40ba694850 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81be471d87be456caec2a5384f121ac2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 38877 } } peers { permanent_uuid: "c14cba3431e240429e15842b4736a423" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 38047 } } peers { permanent_uuid: "72cefb274fb446f5a898cc40ba694850" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 41553 } }
I20260929 10:53:53.514559 5469 catalog_manager.cc:5495] T aa096bb1e72f44559605f6d78e19b339 P 72cefb274fb446f5a898cc40ba694850 reported cstate change: term changed from 0 to 1, leader changed from <none> to 72cefb274fb446f5a898cc40ba694850 (127.20.80.193). New cstate: current_term: 1 leader_uuid: "72cefb274fb446f5a898cc40ba694850" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81be471d87be456caec2a5384f121ac2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 38877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c14cba3431e240429e15842b4736a423" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 38047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72cefb274fb446f5a898cc40ba694850" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 41553 } health_report { overall_health: HEALTHY } } }
I20260929 10:53:53.534744 5946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:53.532111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46322 (local address 127.20.80.193:41553)
0929 10:53:53.532169 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:53.532171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:53.532182 (+ 11us) server_negotiation.cc:407] Connection header received
0929 10:53:53.532275 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:53.532276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:53.532298 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:53.532414 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:53:53.532553 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.533091 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.533891 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.534092 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.534184 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:53.534369 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:53:53.534479 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:53:53.534580 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:53.534610 (+ 30us) server_negotiation.cc:299] Negotiation successful
0929 10:53:53.534632 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260929 10:53:53.536651 5749 raft_consensus.cc:1261] T aa096bb1e72f44559605f6d78e19b339 P 81be471d87be456caec2a5384f121ac2 [term 1 FOLLOWER]: Refusing update from remote peer 72cefb274fb446f5a898cc40ba694850: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:53:53.536685 5884 raft_consensus.cc:1261] T aa096bb1e72f44559605f6d78e19b339 P c14cba3431e240429e15842b4736a423 [term 1 FOLLOWER]: Refusing update from remote peer 72cefb274fb446f5a898cc40ba694850: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:53:53.537062 5957 consensus_queue.cc:1035] T aa096bb1e72f44559605f6d78e19b339 P 72cefb274fb446f5a898cc40ba694850 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81be471d87be456caec2a5384f121ac2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 38877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:53:53.537195 5954 consensus_queue.cc:1035] T aa096bb1e72f44559605f6d78e19b339 P 72cefb274fb446f5a898cc40ba694850 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c14cba3431e240429e15842b4736a423" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 38047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:53:53.539906 5961 mvcc.cc:205] Tried to move back new op lower bound from 7334622140563292160 to 7334622140472565760. Current Snapshot: MvccSnapshot[applied={T|T < 7334622140563292160}]
I20260929 10:53:53.541839 5469 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43848:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:53:53.541927 5469 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43848:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:53:53.542716 5469 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dd7fa7bed23748beb09c2dbc76c17772: Sending DeleteTablet for 3 replicas of tablet aa096bb1e72f44559605f6d78e19b339
I20260929 10:53:53.543066 5864 tablet_service.cc:1514] Processing DeleteTablet for tablet aa096bb1e72f44559605f6d78e19b339 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:53:53 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:55354
I20260929 10:53:53.543064 5729 tablet_service.cc:1514] Processing DeleteTablet for tablet aa096bb1e72f44559605f6d78e19b339 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:53:53 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:54450
I20260929 10:53:53.543330 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 5531
I20260929 10:53:53.543358 5594 tablet_service.cc:1514] Processing DeleteTablet for tablet aa096bb1e72f44559605f6d78e19b339 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:53:53 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:46320
I20260929 10:53:53.548861 5970 tablet_replica.cc:330] T aa096bb1e72f44559605f6d78e19b339 P 81be471d87be456caec2a5384f121ac2: stopping tablet replica
W20260929 10:53:53.548887 5453 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.80.193:41553 (error 108)
I20260929 10:53:53.549073 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 5666
I20260929 10:53:53.550174 5969 tablet_replica.cc:330] T aa096bb1e72f44559605f6d78e19b339 P c14cba3431e240429e15842b4736a423: stopping tablet replica
I20260929 10:53:53.550302 5969 raft_consensus.cc:2229] T aa096bb1e72f44559605f6d78e19b339 P c14cba3431e240429e15842b4736a423 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:53:53.550428 5969 raft_consensus.cc:2258] T aa096bb1e72f44559605f6d78e19b339 P c14cba3431e240429e15842b4736a423 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:53:53.550908 5969 ts_tablet_manager.cc:1905] T aa096bb1e72f44559605f6d78e19b339 P c14cba3431e240429e15842b4736a423: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:53:53.552152 5941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:53.551710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:41553 (local address 127.0.0.1:46324)
0929 10:53:53.552075 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:53.552116 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.193:41553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345}
W20260929 10:53:53.552280 5453 catalog_manager.cc:4584] TS 72cefb274fb446f5a898cc40ba694850 (127.20.80.193:41553): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet aa096bb1e72f44559605f6d78e19b339: Network error: Client connection negotiation failed: client connection to 127.20.80.193:41553: connect: Connection refused (error 111)
I20260929 10:53:53.552547 5969 ts_tablet_manager.cc:1918] T aa096bb1e72f44559605f6d78e19b339 P c14cba3431e240429e15842b4736a423: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:53:53.552639 5969 log.cc:1196] T aa096bb1e72f44559605f6d78e19b339 P c14cba3431e240429e15842b4736a423: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790678996126930-20803-0/minicluster-data/ts-2/wal/wals/aa096bb1e72f44559605f6d78e19b339
I20260929 10:53:53.552886 5969 ts_tablet_manager.cc:1939] T aa096bb1e72f44559605f6d78e19b339 P c14cba3431e240429e15842b4736a423: Deleting consensus metadata
I20260929 10:53:53.553331 5455 catalog_manager.cc:4841] TS c14cba3431e240429e15842b4736a423 (127.20.80.195:38047): tablet aa096bb1e72f44559605f6d78e19b339 (table test-table [id=794cd58b28134b5c84432d17a14ac46a]) successfully deleted
I20260929 10:53:53.553990 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 5801
I20260929 10:53:53.554018 5941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:53.553893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:38877 (local address 127.0.0.1:54466)
0929 10:53:53.553963 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:53.553995 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.194:38877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260929 10:53:53.559662 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 5436
2026-09-29T10:53:53Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1995 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-29T10:53:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:53:53Z Disabled control of system clock
I20260929 10:53:53.587845 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:36769
--webserver_interface=127.20.80.254
--webserver_port=0
--builtin_ntp_servers=127.20.80.212:45009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:36769
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 10:53:53.668817 5975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:53:53.669059 5975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:53:53.669080 5975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:53:53.670271 5975 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:53:53.670315 5975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:53:53.670327 5975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:53:53.670341 5975 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:53:53.670353 5975 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:53:53.671772 5975 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:45009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:36769
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:36769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:53:53.672013 5975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:53:53.672328 5975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:53:53.677193 5980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:53.677166 5983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:53.677399 5981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:53:53.677490 5975 server_base.cc:1034] running on GCE node
I20260929 10:53:53.677728 5975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:53:53.678006 5975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:53:53.679175 5975 hybrid_clock.cc:648] HybridClock initialized: now 1790679233679147 us; error 68 us; skew 500 ppm
I20260929 10:53:53.680414 5975 webserver.cc:460] Webserver started at http://127.20.80.254:44937/ using document root <none> and password file <none>
I20260929 10:53:53.680663 5975 fs_manager.cc:359] Metadata directory not provided
I20260929 10:53:53.680732 5975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:53:53.680886 5975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:53:53.682022 5975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "2b090b1412ef4849a4a79a54f3bc8c54"
format_stamp: "Formatted at 2026-09-29 10:53:53 on dist-test-slave-vqch"
server_key: "122d0322d765a5c653036a2b70d9992c"
server_key_iv: "548814c5eb93d2e1f70a9ab39c14a191"
server_key_version: "encryptionkey@0"
I20260929 10:53:53.682386 5975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "2b090b1412ef4849a4a79a54f3bc8c54"
format_stamp: "Formatted at 2026-09-29 10:53:53 on dist-test-slave-vqch"
server_key: "122d0322d765a5c653036a2b70d9992c"
server_key_iv: "548814c5eb93d2e1f70a9ab39c14a191"
server_key_version: "encryptionkey@0"
I20260929 10:53:53.683768 5975 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 10:53:53.684492 5989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:53.684674 5975 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:53:53.684770 5975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "2b090b1412ef4849a4a79a54f3bc8c54"
format_stamp: "Formatted at 2026-09-29 10:53:53 on dist-test-slave-vqch"
server_key: "122d0322d765a5c653036a2b70d9992c"
server_key_iv: "548814c5eb93d2e1f70a9ab39c14a191"
server_key_version: "encryptionkey@0"
I20260929 10:53:53.684839 5975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:53:53.717092 5975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:53:53.717399 5975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:53:53.717506 5975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:53:53.721587 5975 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:36769
I20260929 10:53:53.721700 6043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:36769 every 8 connection(s)
I20260929 10:53:53.722021 5975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:53:53.722481 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 5975
I20260929 10:53:53.722569 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:53:53.722780 20803 external_mini_cluster.cc:1428] Setting key 38072908fd4f8fec792940015af3b306
I20260929 10:53:53.723354 6044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:53:53.726354 6044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b090b1412ef4849a4a79a54f3bc8c54: Bootstrap starting.
I20260929 10:53:53.727027 6044 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b090b1412ef4849a4a79a54f3bc8c54: Neither blocks nor log segments found. Creating new log.
I20260929 10:53:53.727386 6044 log.cc:824] T 00000000000000000000000000000000 P 2b090b1412ef4849a4a79a54f3bc8c54: Log is configured to *not* fsync() on all Append() calls
I20260929 10:53:53.728194 6044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b090b1412ef4849a4a79a54f3bc8c54: No bootstrap required, opened a new log
I20260929 10:53:53.728437 6047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:53.723380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36544 (local address 127.20.80.254:36769)
0929 10:53:53.723601 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:53.723609 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:53.723634 (+ 25us) server_negotiation.cc:407] Connection header received
0929 10:53:53.724078 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:53.724084 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:53.724143 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:53.724308 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:53:53.724520 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.725480 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.726279 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.726740 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.726895 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:53.727722 (+ 827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:53:53.727726 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:53:53.727726 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:53:53.727896 (+ 170us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:53:53.727936 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:53:53.727947 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:53:53.728025 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:53.728102 (+ 77us) server_negotiation.cc:299] Negotiation successful
0929 10:53:53.728192 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":103,"threads_started":1}
I20260929 10:53:53.730423 6044 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b090b1412ef4849a4a79a54f3bc8c54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b090b1412ef4849a4a79a54f3bc8c54" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 36769 } }
I20260929 10:53:53.730566 6044 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b090b1412ef4849a4a79a54f3bc8c54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:53:53.730592 6044 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b090b1412ef4849a4a79a54f3bc8c54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b090b1412ef4849a4a79a54f3bc8c54, State: Initialized, Role: FOLLOWER
I20260929 10:53:53.730772 6044 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b090b1412ef4849a4a79a54f3bc8c54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b090b1412ef4849a4a79a54f3bc8c54" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 36769 } }
I20260929 10:53:53.730840 6044 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b090b1412ef4849a4a79a54f3bc8c54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:53:53.730885 6044 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b090b1412ef4849a4a79a54f3bc8c54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:53:53.730938 6044 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b090b1412ef4849a4a79a54f3bc8c54 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:53:53.731575 6044 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b090b1412ef4849a4a79a54f3bc8c54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b090b1412ef4849a4a79a54f3bc8c54" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 36769 } }
I20260929 10:53:53.731721 6044 leader_election.cc:302] T 00000000000000000000000000000000 P 2b090b1412ef4849a4a79a54f3bc8c54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b090b1412ef4849a4a79a54f3bc8c54; no voters:
I20260929 10:53:53.731911 6044 leader_election.cc:288] T 00000000000000000000000000000000 P 2b090b1412ef4849a4a79a54f3bc8c54 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:53:53.731982 6049 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b090b1412ef4849a4a79a54f3bc8c54 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:53:53.732143 6049 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b090b1412ef4849a4a79a54f3bc8c54 [term 1 LEADER]: Becoming Leader. State: Replica: 2b090b1412ef4849a4a79a54f3bc8c54, State: Running, Role: LEADER
I20260929 10:53:53.732220 6044 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b090b1412ef4849a4a79a54f3bc8c54 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:53:53.732244 6049 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b090b1412ef4849a4a79a54f3bc8c54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b090b1412ef4849a4a79a54f3bc8c54" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 36769 } }
I20260929 10:53:53.732622 6050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b090b1412ef4849a4a79a54f3bc8c54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b090b1412ef4849a4a79a54f3bc8c54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b090b1412ef4849a4a79a54f3bc8c54" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 36769 } } }
I20260929 10:53:53.732630 6051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b090b1412ef4849a4a79a54f3bc8c54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b090b1412ef4849a4a79a54f3bc8c54. Latest consensus state: current_term: 1 leader_uuid: "2b090b1412ef4849a4a79a54f3bc8c54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b090b1412ef4849a4a79a54f3bc8c54" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 36769 } } }
I20260929 10:53:53.732780 6050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b090b1412ef4849a4a79a54f3bc8c54 [sys.catalog]: This master's current role is: LEADER
I20260929 10:53:53.732823 6051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b090b1412ef4849a4a79a54f3bc8c54 [sys.catalog]: This master's current role is: LEADER
I20260929 10:53:53.733279 6058 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:53:53.733767 6058 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:53:53.735244 6058 catalog_manager.cc:1347] Generated new cluster ID: 761a6ae0c77a48f7842b5bc26fa35726
I20260929 10:53:53.735303 6058 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:53:53.748201 6058 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:53:53.748720 6058 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:53:53.755313 6058 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b090b1412ef4849a4a79a54f3bc8c54: Generated new TSK 0
I20260929 10:53:53.755505 6058 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:53:53.761634 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.193:0
--local_ip_for_outbound_sockets=127.20.80.193
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:36769
--builtin_ntp_servers=127.20.80.212:45009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 10:53:53.851438 6068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:53:53.851668 6068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:53:53.851707 6068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:53:53.852913 6068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:53:53.852977 6068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:53:53.853039 6068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.193
I20260929 10:53:53.854398 6068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:45009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:36769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.193
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:53:53.854637 6068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:53:53.854954 6068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:53:53.861078 6074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:53.861073 6073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:53.861073 6076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:53:53.861361 6068 server_base.cc:1034] running on GCE node
I20260929 10:53:53.861586 6068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:53:53.861843 6068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:53:53.863039 6068 hybrid_clock.cc:648] HybridClock initialized: now 1790679233863019 us; error 49 us; skew 500 ppm
I20260929 10:53:53.864511 6068 webserver.cc:460] Webserver started at http://127.20.80.193:43679/ using document root <none> and password file <none>
I20260929 10:53:53.864784 6068 fs_manager.cc:359] Metadata directory not provided
I20260929 10:53:53.864858 6068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:53:53.865064 6068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:53:53.866159 6068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-0/data/instance:
uuid: "eb7737201f3b4c7e995b519045b57427"
format_stamp: "Formatted at 2026-09-29 10:53:53 on dist-test-slave-vqch"
server_key: "df1c7314366aed71836404a1e7dce416"
server_key_iv: "36bc8e98c4d5c659b3b8fd6d1d3a2796"
server_key_version: "encryptionkey@0"
I20260929 10:53:53.866526 6068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance:
uuid: "eb7737201f3b4c7e995b519045b57427"
format_stamp: "Formatted at 2026-09-29 10:53:53 on dist-test-slave-vqch"
server_key: "df1c7314366aed71836404a1e7dce416"
server_key_iv: "36bc8e98c4d5c659b3b8fd6d1d3a2796"
server_key_version: "encryptionkey@0"
I20260929 10:53:53.867890 6068 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:53:53.868773 6082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:53.868950 6068 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:53:53.869071 6068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-0/wal
uuid: "eb7737201f3b4c7e995b519045b57427"
format_stamp: "Formatted at 2026-09-29 10:53:53 on dist-test-slave-vqch"
server_key: "df1c7314366aed71836404a1e7dce416"
server_key_iv: "36bc8e98c4d5c659b3b8fd6d1d3a2796"
server_key_version: "encryptionkey@0"
I20260929 10:53:53.869175 6068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:53:53.883064 6068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:53:53.883394 6068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:53:53.883548 6068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:53:53.883780 6068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:53:53.884104 6068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:53:53.884166 6068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:53.884231 6068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:53:53.884261 6068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:53.890195 6068 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.193:36633
I20260929 10:53:53.890234 6197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.193:36633 every 8 connection(s)
I20260929 10:53:53.890582 6068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
I20260929 10:53:53.895788 6047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:53.891331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:41227 (local address 127.20.80.254:36769)
0929 10:53:53.891396 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:53.891399 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:53.891658 (+ 259us) server_negotiation.cc:407] Connection header received
0929 10:53:53.892139 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:53.892150 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:53.892162 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:53.892216 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:53:53.892851 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.893418 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.894424 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.894612 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.894673 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:53.895579 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:53:53.895580 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:53:53.895580 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:53:53.895584 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:53:53.895597 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:53:53.895600 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:53:53.895642 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:53.895705 (+ 63us) server_negotiation.cc:299] Negotiation successful
0929 10:53:53.895730 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 10:53:53.895983 6200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:53.891341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:36769 (local address 127.20.80.193:41227)
0929 10:53:53.891540 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:53.891574 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:53.892057 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:53.892274 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:53.892277 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:53.892284 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:53:53.892783 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:53.892787 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.893511 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.893512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:53.894338 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:53.894341 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:53.894749 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:53.894750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:53.894866 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:53.895451 (+ 585us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:53:53.895451 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:53:53.895479 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:53:53.895501 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:53:53.895642 (+ 141us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:53:53.895644 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:53:53.895647 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:53:53.895693 (+ 46us) client_negotiation.cc:240] Negotiation successful
0929 10:53:53.895774 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":82,"threads_started":1}
I20260929 10:53:53.896502 6198 heartbeater.cc:347] Connected to a master server at 127.20.80.254:36769
I20260929 10:53:53.896582 6198 heartbeater.cc:464] Registering TS with master...
I20260929 10:53:53.896766 6198 heartbeater.cc:511] Master 127.20.80.254:36769 requested a full tablet report, sending...
I20260929 10:53:53.897315 6007 ts_manager.cc:194] Registered new tserver with Master: eb7737201f3b4c7e995b519045b57427 (127.20.80.193:36633)
I20260929 10:53:53.897936 6007 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.80.193:41227
I20260929 10:53:53.898207 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 6068
I20260929 10:53:53.898316 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance
I20260929 10:53:53.898519 20803 external_mini_cluster.cc:1428] Setting key f536593e1c40c75ba94e2e8bcdf6ce3c
I20260929 10:53:53.899411 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.194:0
--local_ip_for_outbound_sockets=127.20.80.194
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:36769
--builtin_ntp_servers=127.20.80.212:45009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 10:53:53.983807 6201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:53:53.983999 6201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:53:53.984020 6201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:53:53.985165 6201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:53:53.985203 6201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:53:53.985227 6201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.194
I20260929 10:53:53.986477 6201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:45009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:36769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.194
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:53:53.986662 6201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:53:53.986905 6201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:53:53.991994 6207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:53.992007 6206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:53.991987 6209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:53:53.992311 6201 server_base.cc:1034] running on GCE node
I20260929 10:53:53.992543 6201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:53:53.993067 6201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:53:53.994220 6201 hybrid_clock.cc:648] HybridClock initialized: now 1790679233994209 us; error 40 us; skew 500 ppm
I20260929 10:53:53.995476 6201 webserver.cc:460] Webserver started at http://127.20.80.194:44635/ using document root <none> and password file <none>
I20260929 10:53:53.995723 6201 fs_manager.cc:359] Metadata directory not provided
I20260929 10:53:53.995796 6201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:53:53.995967 6201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:53:53.997061 6201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-1/data/instance:
uuid: "cc98ae13140946deb7c4e9eb245bb71a"
format_stamp: "Formatted at 2026-09-29 10:53:53 on dist-test-slave-vqch"
server_key: "a1574024d4656b091104100ca276d805"
server_key_iv: "0508e2b68a46a3e1ced35b9a911d5b78"
server_key_version: "encryptionkey@0"
I20260929 10:53:53.997427 6201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance:
uuid: "cc98ae13140946deb7c4e9eb245bb71a"
format_stamp: "Formatted at 2026-09-29 10:53:53 on dist-test-slave-vqch"
server_key: "a1574024d4656b091104100ca276d805"
server_key_iv: "0508e2b68a46a3e1ced35b9a911d5b78"
server_key_version: "encryptionkey@0"
I20260929 10:53:53.998754 6201 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:53:53.999538 6215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:53.999727 6201 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:53:53.999842 6201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-1/wal
uuid: "cc98ae13140946deb7c4e9eb245bb71a"
format_stamp: "Formatted at 2026-09-29 10:53:53 on dist-test-slave-vqch"
server_key: "a1574024d4656b091104100ca276d805"
server_key_iv: "0508e2b68a46a3e1ced35b9a911d5b78"
server_key_version: "encryptionkey@0"
I20260929 10:53:53.999925 6201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:53:54.015784 6201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:53:54.016113 6201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:53:54.016269 6201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:53:54.016507 6201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:53:54.016847 6201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:53:54.016901 6201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:54.016933 6201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:53:54.016961 6201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:54.022842 6201 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.194:43417
I20260929 10:53:54.022891 6329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.194:43417 every 8 connection(s)
I20260929 10:53:54.023202 6201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
I20260929 10:53:54.024101 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 6201
I20260929 10:53:54.024189 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance
I20260929 10:53:54.024382 20803 external_mini_cluster.cc:1428] Setting key 8b7d6a0efe4f41233b2e3a26885cf22f
I20260929 10:53:54.025516 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.195:0
--local_ip_for_outbound_sockets=127.20.80.195
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:36769
--builtin_ntp_servers=127.20.80.212:45009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 10:53:54.029093 6047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:54.023921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:55243 (local address 127.20.80.254:36769)
0929 10:53:54.023994 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:54.023996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:54.024231 (+ 235us) server_negotiation.cc:407] Connection header received
0929 10:53:54.024952 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:54.024954 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:54.024968 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:54.025034 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:53:54.025969 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:54.026475 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:54.027617 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:54.027834 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:54.027922 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:54.028786 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:53:54.028788 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:53:54.028788 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:53:54.028793 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:53:54.028808 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:53:54.028813 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:53:54.028860 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:54.028953 (+ 93us) server_negotiation.cc:299] Negotiation successful
0929 10:53:54.029012 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 10:53:54.029305 6332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:54.023925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:36769 (local address 127.20.80.194:55243)
0929 10:53:54.024103 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:54.024136 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:54.024835 (+ 699us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:54.025121 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:54.025126 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:54.025133 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:53:54.025872 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:54.025879 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:54.026571 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:54.026572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:54.027500 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:54.027503 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:54.027933 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:54.027935 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:54.028064 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:54.028629 (+ 565us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:53:54.028630 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:53:54.028657 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:53:54.028679 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:53:54.028859 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:53:54.028865 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:53:54.028868 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:53:54.028958 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 10:53:54.029080 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":93,"threads_started":1}
I20260929 10:53:54.029767 6330 heartbeater.cc:347] Connected to a master server at 127.20.80.254:36769
I20260929 10:53:54.029881 6330 heartbeater.cc:464] Registering TS with master...
I20260929 10:53:54.030103 6330 heartbeater.cc:511] Master 127.20.80.254:36769 requested a full tablet report, sending...
I20260929 10:53:54.030514 6007 ts_manager.cc:194] Registered new tserver with Master: cc98ae13140946deb7c4e9eb245bb71a (127.20.80.194:43417)
I20260929 10:53:54.030910 6007 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.80.194:55243
W20260929 10:53:54.108816 6333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:53:54.109081 6333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:53:54.109104 6333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:53:54.110307 6333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:53:54.110354 6333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:53:54.110381 6333 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.195
I20260929 10:53:54.111747 6333 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:45009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:36769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.195
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:53:54.112048 6333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:53:54.112406 6333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:53:54.117614 6338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:54.117605 6341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:54.117605 6339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:53:54.117640 6333 server_base.cc:1034] running on GCE node
I20260929 10:53:54.118065 6333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:53:54.118283 6333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:53:54.119439 6333 hybrid_clock.cc:648] HybridClock initialized: now 1790679234119425 us; error 27 us; skew 500 ppm
I20260929 10:53:54.120853 6333 webserver.cc:460] Webserver started at http://127.20.80.195:36433/ using document root <none> and password file <none>
I20260929 10:53:54.121193 6333 fs_manager.cc:359] Metadata directory not provided
I20260929 10:53:54.121273 6333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:53:54.121488 6333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:53:54.122660 6333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-2/data/instance:
uuid: "ca4ccab827fa482c9205a5bb305c2239"
format_stamp: "Formatted at 2026-09-29 10:53:54 on dist-test-slave-vqch"
server_key: "f3f3a8589cc0692b69ab92cd8969f4bb"
server_key_iv: "02f216265328cd93614fc8ceb5cd9869"
server_key_version: "encryptionkey@0"
I20260929 10:53:54.123049 6333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance:
uuid: "ca4ccab827fa482c9205a5bb305c2239"
format_stamp: "Formatted at 2026-09-29 10:53:54 on dist-test-slave-vqch"
server_key: "f3f3a8589cc0692b69ab92cd8969f4bb"
server_key_iv: "02f216265328cd93614fc8ceb5cd9869"
server_key_version: "encryptionkey@0"
I20260929 10:53:54.124492 6333 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:53:54.125500 6347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:54.125765 6333 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 10:53:54.125885 6333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-2/wal
uuid: "ca4ccab827fa482c9205a5bb305c2239"
format_stamp: "Formatted at 2026-09-29 10:53:54 on dist-test-slave-vqch"
server_key: "f3f3a8589cc0692b69ab92cd8969f4bb"
server_key_iv: "02f216265328cd93614fc8ceb5cd9869"
server_key_version: "encryptionkey@0"
I20260929 10:53:54.125964 6333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:53:54.143656 6333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:53:54.144019 6333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:53:54.144178 6333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:53:54.144433 6333 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:53:54.144771 6333 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:53:54.144829 6333 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:54.144869 6333 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:53:54.144896 6333 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:54.150674 6333 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.195:45253
I20260929 10:53:54.150719 6461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.195:45253 every 8 connection(s)
I20260929 10:53:54.151052 6333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
I20260929 10:53:54.156602 6047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:54.151848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:40883 (local address 127.20.80.254:36769)
0929 10:53:54.151941 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:54.151943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:54.152058 (+ 115us) server_negotiation.cc:407] Connection header received
0929 10:53:54.152609 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:54.152610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:54.152622 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:54.152678 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:53:54.153361 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:54.153853 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:54.154977 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:54.155169 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:54.155227 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:54.156319 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:53:54.156320 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:53:54.156320 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:53:54.156325 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:53:54.156341 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:53:54.156344 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:53:54.156392 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:54.156518 (+ 126us) server_negotiation.cc:299] Negotiation successful
0929 10:53:54.156540 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 10:53:54.156895 6464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:54.151771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:36769 (local address 127.20.80.195:40883)
0929 10:53:54.151953 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:54.151988 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:54.152512 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:54.152739 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:54.152743 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:54.152751 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:53:54.153287 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:54.153291 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:54.153966 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:54.153967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:54.154889 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:54.154891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:54.155307 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:54.155308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:54.155448 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:54.156156 (+ 708us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:53:54.156157 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:53:54.156188 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:53:54.156214 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:53:54.156429 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:53:54.156435 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:53:54.156438 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:53:54.156511 (+ 73us) client_negotiation.cc:240] Negotiation successful
0929 10:53:54.156602 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
I20260929 10:53:54.157435 6462 heartbeater.cc:347] Connected to a master server at 127.20.80.254:36769
I20260929 10:53:54.157526 6462 heartbeater.cc:464] Registering TS with master...
I20260929 10:53:54.157683 6462 heartbeater.cc:511] Master 127.20.80.254:36769 requested a full tablet report, sending...
I20260929 10:53:54.158048 6007 ts_manager.cc:194] Registered new tserver with Master: ca4ccab827fa482c9205a5bb305c2239 (127.20.80.195:45253)
I20260929 10:53:54.158397 6007 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.80.195:40883
I20260929 10:53:54.160564 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 6333
I20260929 10:53:54.160645 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance
I20260929 10:53:54.160837 20803 external_mini_cluster.cc:1428] Setting key d9d98272b6ea43014381b8e7a343de91
I20260929 10:53:54.161686 20803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:53:54.166472 6047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:54.163142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36548 (local address 127.20.80.254:36769)
0929 10:53:54.163197 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:54.163199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:54.163331 (+ 132us) server_negotiation.cc:407] Connection header received
0929 10:53:54.163390 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:54.163391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:54.163399 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:54.163453 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:53:54.163582 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:54.164056 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:54.164961 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:54.165204 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:54.165253 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:54.166253 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:53:54.166254 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:53:54.166255 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:53:54.166260 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:53:54.166276 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:53:54.166281 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:53:54.166346 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:54.166389 (+ 43us) server_negotiation.cc:299] Negotiation successful
0929 10:53:54.166412 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260929 10:53:54.168305 6007 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36548:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:53:54.170120 6007 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:53:54.176513 6473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:54.172738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57780 (local address 127.20.80.195:45253)
0929 10:53:54.173126 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:54.173131 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:54.173141 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:53:54.173183 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:54.173185 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:54.173203 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:54.173468 (+ 265us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:53:54.173524 (+ 56us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:54.174299 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:54.175451 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:54.176084 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:54.176159 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:54.176257 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:54.176384 (+ 127us) server_negotiation.cc:299] Negotiation successful
0929 10:53:54.176422 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":103,"threads_started":1}
I20260929 10:53:54.176514 6472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:54.172646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:45253 (local address 127.0.0.1:57780)
0929 10:53:54.172938 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:54.173000 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:54.173067 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:54.173308 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:54.173310 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:54.173313 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:53:54.173378 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:54.173380 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:54.174390 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:54.174391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:54.175362 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:54.175364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:54.176181 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:54.176182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:54.176269 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:54.176286 (+ 17us) client_negotiation.cc:769] Sending connection context
0929 10:53:54.176374 (+ 88us) client_negotiation.cc:240] Negotiation successful
0929 10:53:54.176426 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":54,"threads_started":1}
I20260929 10:53:54.177291 6470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:54.172698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:36633 (local address 127.0.0.1:43820)
0929 10:53:54.173178 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:54.173186 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:54.173256 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:54.173647 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:54.173648 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:54.173650 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:53:54.173684 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:54.173685 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:54.174635 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:54.174636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:54.175943 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:54.175947 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:54.177038 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:54.177039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:54.177105 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:54.177108 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:53:54.177177 (+ 69us) client_negotiation.cc:240] Negotiation successful
0929 10:53:54.177223 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"mutex_wait_us":2,"thread_start_us":78,"threads_started":1}
I20260929 10:53:54.177328 6474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:54.172905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43820 (local address 127.20.80.193:36633)
0929 10:53:54.173466 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:54.173470 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:54.173481 (+ 11us) server_negotiation.cc:407] Connection header received
0929 10:53:54.173531 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:54.173533 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:54.173552 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:54.173646 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:53:54.173757 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:54.174502 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:54.176046 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:54.176801 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:54.176878 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:54.176957 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:54.177195 (+ 238us) server_negotiation.cc:299] Negotiation successful
0929 10:53:54.177248 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":107,"threads_started":1}
I20260929 10:53:54.177407 6395 tablet_service.cc:1467] Processing CreateTablet for tablet 3f35d51b6da043fea19a61623ead0e35 (DEFAULT_TABLE table=test-table [id=400e84577ce24f3f94c08059a4853ea2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:53:54.177829 6395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f35d51b6da043fea19a61623ead0e35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:53:54.178097 6131 tablet_service.cc:1467] Processing CreateTablet for tablet 3f35d51b6da043fea19a61623ead0e35 (DEFAULT_TABLE table=test-table [id=400e84577ce24f3f94c08059a4853ea2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:53:54.178520 6131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f35d51b6da043fea19a61623ead0e35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:53:54.179508 6471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:54.172665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:43417 (local address 127.0.0.1:44012)
0929 10:53:54.172947 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:54.173012 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:54.173078 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:54.174806 (+ 1728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:54.174807 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:54.174810 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:53:54.174846 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:54.174848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:54.176231 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:54.176232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:54.178244 (+ 2012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:54.178248 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:54.179267 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:54.179268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:54.179335 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:54.179339 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:53:54.179399 (+ 60us) client_negotiation.cc:240] Negotiation successful
0929 10:53:54.179438 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":10,"thread_start_us":105,"threads_started":1}
I20260929 10:53:54.179564 6475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:54.172938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44012 (local address 127.20.80.194:43417)
0929 10:53:54.173223 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:54.173234 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:54.173266 (+ 32us) server_negotiation.cc:407] Connection header received
0929 10:53:54.174671 (+ 1405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:54.174675 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:54.174699 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:54.174795 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:53:54.174917 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:54.176089 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:54.178369 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:54.179155 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:54.179252 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:54.179349 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:54.179416 (+ 67us) server_negotiation.cc:299] Negotiation successful
0929 10:53:54.179466 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":140,"threads_started":1}
I20260929 10:53:54.180600 6263 tablet_service.cc:1467] Processing CreateTablet for tablet 3f35d51b6da043fea19a61623ead0e35 (DEFAULT_TABLE table=test-table [id=400e84577ce24f3f94c08059a4853ea2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:53:54.181042 6263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f35d51b6da043fea19a61623ead0e35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:53:54.181789 6476 tablet_bootstrap.cc:492] T 3f35d51b6da043fea19a61623ead0e35 P eb7737201f3b4c7e995b519045b57427: Bootstrap starting.
I20260929 10:53:54.182417 6476 tablet_bootstrap.cc:654] T 3f35d51b6da043fea19a61623ead0e35 P eb7737201f3b4c7e995b519045b57427: Neither blocks nor log segments found. Creating new log.
I20260929 10:53:54.182711 6476 log.cc:824] T 3f35d51b6da043fea19a61623ead0e35 P eb7737201f3b4c7e995b519045b57427: Log is configured to *not* fsync() on all Append() calls
I20260929 10:53:54.183511 6476 tablet_bootstrap.cc:492] T 3f35d51b6da043fea19a61623ead0e35 P eb7737201f3b4c7e995b519045b57427: No bootstrap required, opened a new log
I20260929 10:53:54.183566 6476 ts_tablet_manager.cc:1397] T 3f35d51b6da043fea19a61623ead0e35 P eb7737201f3b4c7e995b519045b57427: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:53:54.184672 6479 tablet_bootstrap.cc:492] T 3f35d51b6da043fea19a61623ead0e35 P cc98ae13140946deb7c4e9eb245bb71a: Bootstrap starting.
I20260929 10:53:54.185012 6477 tablet_bootstrap.cc:492] T 3f35d51b6da043fea19a61623ead0e35 P ca4ccab827fa482c9205a5bb305c2239: Bootstrap starting.
I20260929 10:53:54.185554 6479 tablet_bootstrap.cc:654] T 3f35d51b6da043fea19a61623ead0e35 P cc98ae13140946deb7c4e9eb245bb71a: Neither blocks nor log segments found. Creating new log.
I20260929 10:53:54.185815 6477 tablet_bootstrap.cc:654] T 3f35d51b6da043fea19a61623ead0e35 P ca4ccab827fa482c9205a5bb305c2239: Neither blocks nor log segments found. Creating new log.
I20260929 10:53:54.185894 6479 log.cc:824] T 3f35d51b6da043fea19a61623ead0e35 P cc98ae13140946deb7c4e9eb245bb71a: Log is configured to *not* fsync() on all Append() calls
I20260929 10:53:54.185808 6476 raft_consensus.cc:348] T 3f35d51b6da043fea19a61623ead0e35 P eb7737201f3b4c7e995b519045b57427 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca4ccab827fa482c9205a5bb305c2239" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 45253 } } peers { permanent_uuid: "cc98ae13140946deb7c4e9eb245bb71a" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 43417 } } peers { permanent_uuid: "eb7737201f3b4c7e995b519045b57427" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 36633 } }
I20260929 10:53:54.185989 6476 raft_consensus.cc:374] T 3f35d51b6da043fea19a61623ead0e35 P eb7737201f3b4c7e995b519045b57427 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:53:54.186033 6476 raft_consensus.cc:729] T 3f35d51b6da043fea19a61623ead0e35 P eb7737201f3b4c7e995b519045b57427 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb7737201f3b4c7e995b519045b57427, State: Initialized, Role: FOLLOWER
I20260929 10:53:54.186136 6477 log.cc:824] T 3f35d51b6da043fea19a61623ead0e35 P ca4ccab827fa482c9205a5bb305c2239: Log is configured to *not* fsync() on all Append() calls
I20260929 10:53:54.186175 6476 consensus_queue.cc:260] T 3f35d51b6da043fea19a61623ead0e35 P eb7737201f3b4c7e995b519045b57427 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca4ccab827fa482c9205a5bb305c2239" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 45253 } } peers { permanent_uuid: "cc98ae13140946deb7c4e9eb245bb71a" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 43417 } } peers { permanent_uuid: "eb7737201f3b4c7e995b519045b57427" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 36633 } }
I20260929 10:53:54.186486 6476 ts_tablet_manager.cc:1428] T 3f35d51b6da043fea19a61623ead0e35 P eb7737201f3b4c7e995b519045b57427: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 10:53:54.186619 6198 heartbeater.cc:503] Master 127.20.80.254:36769 was elected leader, sending a full tablet report...
I20260929 10:53:54.186720 6479 tablet_bootstrap.cc:492] T 3f35d51b6da043fea19a61623ead0e35 P cc98ae13140946deb7c4e9eb245bb71a: No bootstrap required, opened a new log
I20260929 10:53:54.186803 6479 ts_tablet_manager.cc:1397] T 3f35d51b6da043fea19a61623ead0e35 P cc98ae13140946deb7c4e9eb245bb71a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:53:54.186834 6477 tablet_bootstrap.cc:492] T 3f35d51b6da043fea19a61623ead0e35 P ca4ccab827fa482c9205a5bb305c2239: No bootstrap required, opened a new log
I20260929 10:53:54.186919 6477 ts_tablet_manager.cc:1397] T 3f35d51b6da043fea19a61623ead0e35 P ca4ccab827fa482c9205a5bb305c2239: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:53:54.189528 6477 raft_consensus.cc:348] T 3f35d51b6da043fea19a61623ead0e35 P ca4ccab827fa482c9205a5bb305c2239 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca4ccab827fa482c9205a5bb305c2239" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 45253 } } peers { permanent_uuid: "cc98ae13140946deb7c4e9eb245bb71a" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 43417 } } peers { permanent_uuid: "eb7737201f3b4c7e995b519045b57427" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 36633 } }
I20260929 10:53:54.189528 6479 raft_consensus.cc:348] T 3f35d51b6da043fea19a61623ead0e35 P cc98ae13140946deb7c4e9eb245bb71a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca4ccab827fa482c9205a5bb305c2239" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 45253 } } peers { permanent_uuid: "cc98ae13140946deb7c4e9eb245bb71a" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 43417 } } peers { permanent_uuid: "eb7737201f3b4c7e995b519045b57427" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 36633 } }
I20260929 10:53:54.189692 6479 raft_consensus.cc:374] T 3f35d51b6da043fea19a61623ead0e35 P cc98ae13140946deb7c4e9eb245bb71a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:53:54.189692 6477 raft_consensus.cc:374] T 3f35d51b6da043fea19a61623ead0e35 P ca4ccab827fa482c9205a5bb305c2239 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:53:54.189729 6477 raft_consensus.cc:729] T 3f35d51b6da043fea19a61623ead0e35 P ca4ccab827fa482c9205a5bb305c2239 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca4ccab827fa482c9205a5bb305c2239, State: Initialized, Role: FOLLOWER
I20260929 10:53:54.189729 6479 raft_consensus.cc:729] T 3f35d51b6da043fea19a61623ead0e35 P cc98ae13140946deb7c4e9eb245bb71a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc98ae13140946deb7c4e9eb245bb71a, State: Initialized, Role: FOLLOWER
I20260929 10:53:54.189853 6477 consensus_queue.cc:260] T 3f35d51b6da043fea19a61623ead0e35 P ca4ccab827fa482c9205a5bb305c2239 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca4ccab827fa482c9205a5bb305c2239" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 45253 } } peers { permanent_uuid: "cc98ae13140946deb7c4e9eb245bb71a" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 43417 } } peers { permanent_uuid: "eb7737201f3b4c7e995b519045b57427" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 36633 } }
I20260929 10:53:54.189881 6479 consensus_queue.cc:260] T 3f35d51b6da043fea19a61623ead0e35 P cc98ae13140946deb7c4e9eb245bb71a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca4ccab827fa482c9205a5bb305c2239" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 45253 } } peers { permanent_uuid: "cc98ae13140946deb7c4e9eb245bb71a" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 43417 } } peers { permanent_uuid: "eb7737201f3b4c7e995b519045b57427" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 36633 } }
I20260929 10:53:54.190120 6479 ts_tablet_manager.cc:1428] T 3f35d51b6da043fea19a61623ead0e35 P cc98ae13140946deb7c4e9eb245bb71a: Time spent starting tablet: real 0.003s user 0.001s sys 0.003s
I20260929 10:53:54.190120 6477 ts_tablet_manager.cc:1428] T 3f35d51b6da043fea19a61623ead0e35 P ca4ccab827fa482c9205a5bb305c2239: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 10:53:54.190173 6330 heartbeater.cc:503] Master 127.20.80.254:36769 was elected leader, sending a full tablet report...
I20260929 10:53:54.190438 6462 heartbeater.cc:503] Master 127.20.80.254:36769 was elected leader, sending a full tablet report...
W20260929 10:53:54.273751 6331 tablet.cc:2369] T 3f35d51b6da043fea19a61623ead0e35 P cc98ae13140946deb7c4e9eb245bb71a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:53:54.391381 6199 tablet.cc:2369] T 3f35d51b6da043fea19a61623ead0e35 P eb7737201f3b4c7e995b519045b57427: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:53:54.401571 6463 tablet.cc:2369] T 3f35d51b6da043fea19a61623ead0e35 P ca4ccab827fa482c9205a5bb305c2239: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:53:54.529867 6483 raft_consensus.cc:482] T 3f35d51b6da043fea19a61623ead0e35 P ca4ccab827fa482c9205a5bb305c2239 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:53:54.530019 6483 raft_consensus.cc:504] T 3f35d51b6da043fea19a61623ead0e35 P ca4ccab827fa482c9205a5bb305c2239 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca4ccab827fa482c9205a5bb305c2239" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 45253 } } peers { permanent_uuid: "cc98ae13140946deb7c4e9eb245bb71a" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 43417 } } peers { permanent_uuid: "eb7737201f3b4c7e995b519045b57427" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 36633 } }
I20260929 10:53:54.530400 6483 leader_election.cc:288] T 3f35d51b6da043fea19a61623ead0e35 P ca4ccab827fa482c9205a5bb305c2239 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc98ae13140946deb7c4e9eb245bb71a (127.20.80.194:43417), eb7737201f3b4c7e995b519045b57427 (127.20.80.193:36633)
I20260929 10:53:54.533704 6464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:54.530594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:43417 (local address 127.20.80.195:53391)
0929 10:53:54.530661 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:54.530673 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:54.530754 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:54.530942 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:54.530943 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:54.530945 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:53:54.531023 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:54.531026 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:54.531764 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:54.531765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:54.532812 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:54.532815 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:54.533441 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:54.533442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:54.533511 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:54.533515 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:53:54.533579 (+ 64us) client_negotiation.cc:240] Negotiation successful
0929 10:53:54.533624 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260929 10:53:54.533712 6475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:54.530680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:53391 (local address 127.20.80.194:43417)
0929 10:53:54.530731 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:54.530733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:54.530744 (+ 11us) server_negotiation.cc:407] Connection header received
0929 10:53:54.530855 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:54.530856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:54.530867 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:54.530924 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:53:54.531102 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:54.531677 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:54.532891 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:54.533359 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:54.533436 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:54.533443 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:54.533603 (+ 160us) server_negotiation.cc:299] Negotiation successful
0929 10:53:54.533639 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260929 10:53:54.533928 6474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:54.530682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:47269 (local address 127.20.80.193:36633)
0929 10:53:54.530731 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:54.530733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:54.530882 (+ 149us) server_negotiation.cc:407] Connection header received
0929 10:53:54.530958 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:54.530959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:54.530971 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:54.531023 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:53:54.531223 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:54.531904 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:54.533049 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:54.533523 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:54.533596 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:54.533604 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:54.533813 (+ 209us) server_negotiation.cc:299] Negotiation successful
0929 10:53:54.533860 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 10:53:54.534047 6283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f35d51b6da043fea19a61623ead0e35" candidate_uuid: "ca4ccab827fa482c9205a5bb305c2239" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc98ae13140946deb7c4e9eb245bb71a" is_pre_election: true
I20260929 10:53:54.534175 6485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:54.530595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:36633 (local address 127.20.80.195:47269)
0929 10:53:54.530815 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:54.530826 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:54.530891 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:54.531034 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:54.531034 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:54.531035 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:53:54.531121 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:54.531123 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:54.531995 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:54.531996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:54.532931 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:54.532934 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:54.533607 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:54.533608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:54.533703 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:54.533707 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:53:54.534043 (+ 336us) client_negotiation.cc:240] Negotiation successful
0929 10:53:54.534084 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":24,"thread_start_us":116,"threads_started":1}
I20260929 10:53:54.534246 6283 raft_consensus.cc:2454] T 3f35d51b6da043fea19a61623ead0e35 P cc98ae13140946deb7c4e9eb245bb71a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca4ccab827fa482c9205a5bb305c2239 in term 0.
I20260929 10:53:54.534468 6348 leader_election.cc:302] T 3f35d51b6da043fea19a61623ead0e35 P ca4ccab827fa482c9205a5bb305c2239 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca4ccab827fa482c9205a5bb305c2239, cc98ae13140946deb7c4e9eb245bb71a; no voters:
I20260929 10:53:54.534636 6151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f35d51b6da043fea19a61623ead0e35" candidate_uuid: "ca4ccab827fa482c9205a5bb305c2239" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb7737201f3b4c7e995b519045b57427" is_pre_election: true
I20260929 10:53:54.534642 6483 raft_consensus.cc:2781] T 3f35d51b6da043fea19a61623ead0e35 P ca4ccab827fa482c9205a5bb305c2239 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:53:54.534721 6483 raft_consensus.cc:482] T 3f35d51b6da043fea19a61623ead0e35 P ca4ccab827fa482c9205a5bb305c2239 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:53:54.534760 6483 raft_consensus.cc:3037] T 3f35d51b6da043fea19a61623ead0e35 P ca4ccab827fa482c9205a5bb305c2239 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:53:54.534788 6151 raft_consensus.cc:2454] T 3f35d51b6da043fea19a61623ead0e35 P eb7737201f3b4c7e995b519045b57427 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca4ccab827fa482c9205a5bb305c2239 in term 0.
I20260929 10:53:54.535521 6483 raft_consensus.cc:504] T 3f35d51b6da043fea19a61623ead0e35 P ca4ccab827fa482c9205a5bb305c2239 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca4ccab827fa482c9205a5bb305c2239" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 45253 } } peers { permanent_uuid: "cc98ae13140946deb7c4e9eb245bb71a" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 43417 } } peers { permanent_uuid: "eb7737201f3b4c7e995b519045b57427" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 36633 } }
I20260929 10:53:54.535678 6483 leader_election.cc:288] T 3f35d51b6da043fea19a61623ead0e35 P ca4ccab827fa482c9205a5bb305c2239 [CANDIDATE]: Term 1 election: Requested vote from peers cc98ae13140946deb7c4e9eb245bb71a (127.20.80.194:43417), eb7737201f3b4c7e995b519045b57427 (127.20.80.193:36633)
I20260929 10:53:54.535830 6151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f35d51b6da043fea19a61623ead0e35" candidate_uuid: "ca4ccab827fa482c9205a5bb305c2239" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb7737201f3b4c7e995b519045b57427"
I20260929 10:53:54.535907 6283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f35d51b6da043fea19a61623ead0e35" candidate_uuid: "ca4ccab827fa482c9205a5bb305c2239" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc98ae13140946deb7c4e9eb245bb71a"
I20260929 10:53:54.535910 6151 raft_consensus.cc:3037] T 3f35d51b6da043fea19a61623ead0e35 P eb7737201f3b4c7e995b519045b57427 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:53:54.535979 6283 raft_consensus.cc:3037] T 3f35d51b6da043fea19a61623ead0e35 P cc98ae13140946deb7c4e9eb245bb71a [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:53:54.536857 6283 raft_consensus.cc:2454] T 3f35d51b6da043fea19a61623ead0e35 P cc98ae13140946deb7c4e9eb245bb71a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca4ccab827fa482c9205a5bb305c2239 in term 1.
I20260929 10:53:54.536857 6151 raft_consensus.cc:2454] T 3f35d51b6da043fea19a61623ead0e35 P eb7737201f3b4c7e995b519045b57427 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca4ccab827fa482c9205a5bb305c2239 in term 1.
I20260929 10:53:54.537097 6349 leader_election.cc:302] T 3f35d51b6da043fea19a61623ead0e35 P ca4ccab827fa482c9205a5bb305c2239 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca4ccab827fa482c9205a5bb305c2239, eb7737201f3b4c7e995b519045b57427; no voters:
I20260929 10:53:54.537228 6483 raft_consensus.cc:2781] T 3f35d51b6da043fea19a61623ead0e35 P ca4ccab827fa482c9205a5bb305c2239 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:53:54.537386 6483 raft_consensus.cc:686] T 3f35d51b6da043fea19a61623ead0e35 P ca4ccab827fa482c9205a5bb305c2239 [term 1 LEADER]: Becoming Leader. State: Replica: ca4ccab827fa482c9205a5bb305c2239, State: Running, Role: LEADER
I20260929 10:53:54.537490 6483 consensus_queue.cc:237] T 3f35d51b6da043fea19a61623ead0e35 P ca4ccab827fa482c9205a5bb305c2239 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca4ccab827fa482c9205a5bb305c2239" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 45253 } } peers { permanent_uuid: "cc98ae13140946deb7c4e9eb245bb71a" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 43417 } } peers { permanent_uuid: "eb7737201f3b4c7e995b519045b57427" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 36633 } }
I20260929 10:53:54.538300 6007 catalog_manager.cc:5495] T 3f35d51b6da043fea19a61623ead0e35 P ca4ccab827fa482c9205a5bb305c2239 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca4ccab827fa482c9205a5bb305c2239 (127.20.80.195). New cstate: current_term: 1 leader_uuid: "ca4ccab827fa482c9205a5bb305c2239" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca4ccab827fa482c9205a5bb305c2239" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 45253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc98ae13140946deb7c4e9eb245bb71a" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 43417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb7737201f3b4c7e995b519045b57427" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 36633 } health_report { overall_health: UNKNOWN } } }
I20260929 10:53:54.617223 6473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:54.614751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57792 (local address 127.20.80.195:45253)
0929 10:53:54.614850 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:54.614852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:54.614859 (+ 7us) server_negotiation.cc:407] Connection header received
0929 10:53:54.614896 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:54.614897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:54.614914 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:54.615002 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:53:54.615118 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:54.615636 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:54.616404 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:54.616603 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:54.616680 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:54.616843 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:53:54.616942 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:53:54.617068 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:54.617093 (+ 25us) server_negotiation.cc:299] Negotiation successful
0929 10:53:54.617112 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 10:53:54.619181 6283 raft_consensus.cc:1261] T 3f35d51b6da043fea19a61623ead0e35 P cc98ae13140946deb7c4e9eb245bb71a [term 1 FOLLOWER]: Refusing update from remote peer ca4ccab827fa482c9205a5bb305c2239: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:53:54.619181 6151 raft_consensus.cc:1261] T 3f35d51b6da043fea19a61623ead0e35 P eb7737201f3b4c7e995b519045b57427 [term 1 FOLLOWER]: Refusing update from remote peer ca4ccab827fa482c9205a5bb305c2239: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:53:54.619603 6486 consensus_queue.cc:1035] T 3f35d51b6da043fea19a61623ead0e35 P ca4ccab827fa482c9205a5bb305c2239 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc98ae13140946deb7c4e9eb245bb71a" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 43417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:53:54.619772 6483 consensus_queue.cc:1035] T 3f35d51b6da043fea19a61623ead0e35 P ca4ccab827fa482c9205a5bb305c2239 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb7737201f3b4c7e995b519045b57427" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 36633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:53:54.622500 6489 mvcc.cc:205] Tried to move back new op lower bound from 7334622144996843520 to 7334622144666116096. Current Snapshot: MvccSnapshot[applied={T|T < 7334622144996843520}]
I20260929 10:53:54.622624 6491 mvcc.cc:205] Tried to move back new op lower bound from 7334622144996843520 to 7334622144666116096. Current Snapshot: MvccSnapshot[applied={T|T < 7334622144996843520}]
I20260929 10:53:54.623296 6490 mvcc.cc:205] Tried to move back new op lower bound from 7334622144996843520 to 7334622144666116096. Current Snapshot: MvccSnapshot[applied={T|T < 7334622144996843520 or (T in {7334622144996843520})}]
I20260929 10:53:54.625164 6007 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:53:54.625265 6007 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:53:54.626063 6007 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2b090b1412ef4849a4a79a54f3bc8c54: Sending DeleteTablet for 3 replicas of tablet 3f35d51b6da043fea19a61623ead0e35
I20260929 10:53:54.626428 6263 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f35d51b6da043fea19a61623ead0e35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:53:54 UTC) from {username='slave'} at 127.0.0.1:44012
I20260929 10:53:54.626420 6131 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f35d51b6da043fea19a61623ead0e35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:53:54 UTC) from {username='slave'} at 127.0.0.1:43820
I20260929 10:53:54.626462 6395 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f35d51b6da043fea19a61623ead0e35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:53:54 UTC) from {username='slave'} at 127.0.0.1:57780
I20260929 10:53:54.626636 6501 tablet_replica.cc:330] T 3f35d51b6da043fea19a61623ead0e35 P cc98ae13140946deb7c4e9eb245bb71a: stopping tablet replica
I20260929 10:53:54.626730 6501 raft_consensus.cc:2229] T 3f35d51b6da043fea19a61623ead0e35 P cc98ae13140946deb7c4e9eb245bb71a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:53:54.626858 6501 raft_consensus.cc:2258] T 3f35d51b6da043fea19a61623ead0e35 P cc98ae13140946deb7c4e9eb245bb71a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:53:54.626920 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 6068
I20260929 10:53:54.626883 6499 tablet_replica.cc:330] T 3f35d51b6da043fea19a61623ead0e35 P eb7737201f3b4c7e995b519045b57427: stopping tablet replica
I20260929 10:53:54.627013 6500 tablet_replica.cc:330] T 3f35d51b6da043fea19a61623ead0e35 P ca4ccab827fa482c9205a5bb305c2239: stopping tablet replica
I20260929 10:53:54.627103 6500 raft_consensus.cc:2229] T 3f35d51b6da043fea19a61623ead0e35 P ca4ccab827fa482c9205a5bb305c2239 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:53:54.627206 6500 raft_consensus.cc:2258] T 3f35d51b6da043fea19a61623ead0e35 P ca4ccab827fa482c9205a5bb305c2239 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:53:54.627756 6500 ts_tablet_manager.cc:1905] T 3f35d51b6da043fea19a61623ead0e35 P ca4ccab827fa482c9205a5bb305c2239: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:53:54.629367 6501 ts_tablet_manager.cc:1905] T 3f35d51b6da043fea19a61623ead0e35 P cc98ae13140946deb7c4e9eb245bb71a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:53:54.630508 6500 ts_tablet_manager.cc:1918] T 3f35d51b6da043fea19a61623ead0e35 P ca4ccab827fa482c9205a5bb305c2239: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:53:54.630599 6500 log.cc:1196] T 3f35d51b6da043fea19a61623ead0e35 P ca4ccab827fa482c9205a5bb305c2239: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-2/wal/wals/3f35d51b6da043fea19a61623ead0e35
I20260929 10:53:54.630852 6500 ts_tablet_manager.cc:1939] T 3f35d51b6da043fea19a61623ead0e35 P ca4ccab827fa482c9205a5bb305c2239: Deleting consensus metadata
I20260929 10:53:54.630975 6501 ts_tablet_manager.cc:1918] T 3f35d51b6da043fea19a61623ead0e35 P cc98ae13140946deb7c4e9eb245bb71a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:53:54.631088 6501 log.cc:1196] T 3f35d51b6da043fea19a61623ead0e35 P cc98ae13140946deb7c4e9eb245bb71a: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790678996126930-20803-0/minicluster-data/ts-1/wal/wals/3f35d51b6da043fea19a61623ead0e35
I20260929 10:53:54.631364 6501 ts_tablet_manager.cc:1939] T 3f35d51b6da043fea19a61623ead0e35 P cc98ae13140946deb7c4e9eb245bb71a: Deleting consensus metadata
I20260929 10:53:54.631771 5990 catalog_manager.cc:4841] TS cc98ae13140946deb7c4e9eb245bb71a (127.20.80.194:43417): tablet 3f35d51b6da043fea19a61623ead0e35 (table test-table [id=400e84577ce24f3f94c08059a4853ea2]) successfully deleted
W20260929 10:53:54.634958 5991 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.80.193:36633 (error 108)
I20260929 10:53:54.635159 5993 catalog_manager.cc:4841] TS ca4ccab827fa482c9205a5bb305c2239 (127.20.80.195:45253): tablet 3f35d51b6da043fea19a61623ead0e35 (table test-table [id=400e84577ce24f3f94c08059a4853ea2]) successfully deleted
I20260929 10:53:54.635350 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 6201
I20260929 10:53:54.635434 6472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:54.635291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:36633 (local address 127.0.0.1:43824)
0929 10:53:54.635333 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:54.635371 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.193:36633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260929 10:53:54.636371 5991 catalog_manager.cc:4584] TS eb7737201f3b4c7e995b519045b57427 (127.20.80.193:36633): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3f35d51b6da043fea19a61623ead0e35: Network error: Client connection negotiation failed: client connection to 127.20.80.193:36633: connect: Connection refused (error 111)
I20260929 10:53:54.640136 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 6333
I20260929 10:53:54.644776 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 5975
2026-09-29T10:53:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1082 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:53:54 dist-test-slave-vqch krb5kdc[6505](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:53:54 dist-test-slave-vqch krb5kdc[6505](info): set up 2 sockets
Sep 29 10:53:54 dist-test-slave-vqch krb5kdc[6505](info): commencing operation
krb5kdc: starting...
W20260929 10:53:55.676569 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:53:55 dist-test-slave-vqch krb5kdc[6505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:53:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:53:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.80.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
I20260929 10:53:55.800755 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:41547
--webserver_interface=127.20.80.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:35891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:41547
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:53:55.881311 6518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:53:55.881498 6518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:53:55.881522 6518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:53:55.882828 6518 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:53:55.882871 6518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:53:55.882885 6518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:53:55.882898 6518 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:53:55.882910 6518 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:53:55.884346 6518 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:35891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:41547
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:41547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:53:55.884578 6518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:53:55.884804 6518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 10:53:55.889698 6518 server_base.cc:1034] running on GCE node
W20260929 10:53:55.889698 6524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:55.889664 6526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:55.889668 6523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:53:55.890288 6518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:53:55.890628 6518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:53:55.891822 6518 hybrid_clock.cc:648] HybridClock initialized: now 1790679235891834 us; error 74 us; skew 500 ppm
Sep 29 10:53:55 dist-test-slave-vqch krb5kdc[6505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679235, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:53:55.900139 6518 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:53:55.900617 6518 webserver.cc:460] Webserver started at http://127.20.80.254:39139/ using document root <none> and password file <none>
I20260929 10:53:55.900876 6518 fs_manager.cc:359] Metadata directory not provided
I20260929 10:53:55.900951 6518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:53:55.901177 6518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:53:55.902308 6518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "b3b8759eb7fe4af297a0d2061bfd8592"
format_stamp: "Formatted at 2026-09-29 10:53:55 on dist-test-slave-vqch"
server_key: "6bd63e9e86bf8eb1acf18da7c6e5dd1d"
server_key_iv: "e08d42b1089277e2b6e7b225047a4d85"
server_key_version: "encryptionkey@0"
I20260929 10:53:55.902695 6518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "b3b8759eb7fe4af297a0d2061bfd8592"
format_stamp: "Formatted at 2026-09-29 10:53:55 on dist-test-slave-vqch"
server_key: "6bd63e9e86bf8eb1acf18da7c6e5dd1d"
server_key_iv: "e08d42b1089277e2b6e7b225047a4d85"
server_key_version: "encryptionkey@0"
I20260929 10:53:55.904115 6518 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:53:55.904942 6533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:55.905180 6518 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:53:55.905297 6518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "b3b8759eb7fe4af297a0d2061bfd8592"
format_stamp: "Formatted at 2026-09-29 10:53:55 on dist-test-slave-vqch"
server_key: "6bd63e9e86bf8eb1acf18da7c6e5dd1d"
server_key_iv: "e08d42b1089277e2b6e7b225047a4d85"
server_key_version: "encryptionkey@0"
I20260929 10:53:55.905373 6518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:53:55.913681 6518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:53:55.915733 6518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:53:55.915884 6518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:53:55.919950 6518 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:41547
I20260929 10:53:55.920073 6585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:41547 every 8 connection(s)
I20260929 10:53:55.920360 6518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:53:55.921265 6586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:53:55.923487 6586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3b8759eb7fe4af297a0d2061bfd8592: Bootstrap starting.
I20260929 10:53:55.924077 6586 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3b8759eb7fe4af297a0d2061bfd8592: Neither blocks nor log segments found. Creating new log.
I20260929 10:53:55.924378 6586 log.cc:824] T 00000000000000000000000000000000 P b3b8759eb7fe4af297a0d2061bfd8592: Log is configured to *not* fsync() on all Append() calls
I20260929 10:53:55.925161 6586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3b8759eb7fe4af297a0d2061bfd8592: No bootstrap required, opened a new log
I20260929 10:53:55.925449 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 6518
I20260929 10:53:55.925571 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:53:55.925812 20803 external_mini_cluster.cc:1428] Setting key 41fc14b4ac95a49b86dba78deccff737
I20260929 10:53:55.927503 6586 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3b8759eb7fe4af297a0d2061bfd8592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b8759eb7fe4af297a0d2061bfd8592" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 41547 } }
I20260929 10:53:55.927634 6586 raft_consensus.cc:374] T 00000000000000000000000000000000 P b3b8759eb7fe4af297a0d2061bfd8592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:53:55.927657 6586 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3b8759eb7fe4af297a0d2061bfd8592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3b8759eb7fe4af297a0d2061bfd8592, State: Initialized, Role: FOLLOWER
I20260929 10:53:55.927786 6586 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3b8759eb7fe4af297a0d2061bfd8592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b8759eb7fe4af297a0d2061bfd8592" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 41547 } }
I20260929 10:53:55.927858 6586 raft_consensus.cc:388] T 00000000000000000000000000000000 P b3b8759eb7fe4af297a0d2061bfd8592 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:53:55.927881 6586 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3b8759eb7fe4af297a0d2061bfd8592 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:53:55.927949 6586 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3b8759eb7fe4af297a0d2061bfd8592 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:53:55.929116 6586 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3b8759eb7fe4af297a0d2061bfd8592 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b8759eb7fe4af297a0d2061bfd8592" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 41547 } }
I20260929 10:53:55.929221 6586 leader_election.cc:302] T 00000000000000000000000000000000 P b3b8759eb7fe4af297a0d2061bfd8592 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b3b8759eb7fe4af297a0d2061bfd8592; no voters:
I20260929 10:53:55.929359 6586 leader_election.cc:288] T 00000000000000000000000000000000 P b3b8759eb7fe4af297a0d2061bfd8592 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:53:55.929433 6591 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3b8759eb7fe4af297a0d2061bfd8592 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:53:55.929580 6591 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3b8759eb7fe4af297a0d2061bfd8592 [term 1 LEADER]: Becoming Leader. State: Replica: b3b8759eb7fe4af297a0d2061bfd8592, State: Running, Role: LEADER
I20260929 10:53:55.929639 6586 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3b8759eb7fe4af297a0d2061bfd8592 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:53:55.929721 6591 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3b8759eb7fe4af297a0d2061bfd8592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b8759eb7fe4af297a0d2061bfd8592" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 41547 } }
I20260929 10:53:55.930034 6591 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3b8759eb7fe4af297a0d2061bfd8592 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3b8759eb7fe4af297a0d2061bfd8592. Latest consensus state: current_term: 1 leader_uuid: "b3b8759eb7fe4af297a0d2061bfd8592" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b8759eb7fe4af297a0d2061bfd8592" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 41547 } } }
I20260929 10:53:55.930055 6593 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3b8759eb7fe4af297a0d2061bfd8592 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b3b8759eb7fe4af297a0d2061bfd8592" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b8759eb7fe4af297a0d2061bfd8592" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 41547 } } }
I20260929 10:53:55.930202 6593 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3b8759eb7fe4af297a0d2061bfd8592 [sys.catalog]: This master's current role is: LEADER
I20260929 10:53:55.930398 6591 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3b8759eb7fe4af297a0d2061bfd8592 [sys.catalog]: This master's current role is: LEADER
W20260929 10:53:55.931475 6608 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b3b8759eb7fe4af297a0d2061bfd8592: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:53:55.931663 6608 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:53:55.931767 6599 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:53:55.931984 6599 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:53:55.933749 6599 catalog_manager.cc:1347] Generated new cluster ID: cb8d0dad7d05487f8953a0835128a59a
Sep 29 10:53:55 dist-test-slave-vqch krb5kdc[6505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:53:55.933818 6599 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:53:55.938891 6590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:55.926373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38758 (local address 127.20.80.254:41547)
0929 10:53:55.926563 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:55.926572 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:55.926615 (+ 43us) server_negotiation.cc:407] Connection header received
0929 10:53:55.927073 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:55.927082 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:55.927142 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:55.927297 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:53:55.928160 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:55.929601 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:55.930825 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:55.931406 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:55.931540 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:55.934361 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:53:55.934365 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:53:55.934365 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:53:55.934378 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:53:55.936674 (+ 2296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:55.937132 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:55.937136 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:55.937137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:55.937248 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:55.937468 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:55.937469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:55.937469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:55.937810 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:53:55.938441 (+ 631us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:55.938538 (+ 97us) server_negotiation.cc:299] Negotiation successful
0929 10:53:55.938701 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":90,"threads_started":1}
I20260929 10:53:55.987010 6599 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:53:55.987802 6599 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:53:55.993610 6599 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b3b8759eb7fe4af297a0d2061bfd8592: Generated new TSK 0
I20260929 10:53:55.993854 6599 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.80.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
I20260929 10:53:56.035980 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.193:0
--local_ip_for_outbound_sockets=127.20.80.193
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:41547
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:35891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:53:56.134291 6613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:53:56.134500 6613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:53:56.134526 6613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:53:56.135672 6613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:53:56.135717 6613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:53:56.135746 6613 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.193
I20260929 10:53:56.137221 6613 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:35891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:41547
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.193
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:53:56.137488 6613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:53:56.137734 6613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:53:56.142992 6621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:56.143043 6618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:56.142969 6619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:53:56.143100 6613 server_base.cc:1034] running on GCE node
I20260929 10:53:56.143323 6613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:53:56.143571 6613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:53:56.144754 6613 hybrid_clock.cc:648] HybridClock initialized: now 1790679236144756 us; error 42 us; skew 500 ppm
Sep 29 10:53:56 dist-test-slave-vqch krb5kdc[6505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679236, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:53:56.147434 6613 init.cc:377] Logged in from keytab as kudu/127.20.80.193@KRBTEST.COM (short username kudu)
I20260929 10:53:56.147917 6613 webserver.cc:460] Webserver started at http://127.20.80.193:45037/ using document root <none> and password file <none>
I20260929 10:53:56.148157 6613 fs_manager.cc:359] Metadata directory not provided
I20260929 10:53:56.148229 6613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:53:56.148391 6613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:53:56.149552 6613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-0/data/instance:
uuid: "e4700fe2cf334f018612deaa06dff964"
format_stamp: "Formatted at 2026-09-29 10:53:56 on dist-test-slave-vqch"
server_key: "08c8dca62ab43c647688f9e7b5014485"
server_key_iv: "bc4b0b8c9c1cb4e51154e573b1e7ca11"
server_key_version: "encryptionkey@0"
I20260929 10:53:56.149933 6613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance:
uuid: "e4700fe2cf334f018612deaa06dff964"
format_stamp: "Formatted at 2026-09-29 10:53:56 on dist-test-slave-vqch"
server_key: "08c8dca62ab43c647688f9e7b5014485"
server_key_iv: "bc4b0b8c9c1cb4e51154e573b1e7ca11"
server_key_version: "encryptionkey@0"
I20260929 10:53:56.151340 6613 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:53:56.152101 6628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:56.152297 6613 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:53:56.152432 6613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-0/wal
uuid: "e4700fe2cf334f018612deaa06dff964"
format_stamp: "Formatted at 2026-09-29 10:53:56 on dist-test-slave-vqch"
server_key: "08c8dca62ab43c647688f9e7b5014485"
server_key_iv: "bc4b0b8c9c1cb4e51154e573b1e7ca11"
server_key_version: "encryptionkey@0"
I20260929 10:53:56.152508 6613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:53:56.187618 6613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:53:56.189844 6613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:53:56.190021 6613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:53:56.190274 6613 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:53:56.190653 6613 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:53:56.190712 6613 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:56.190757 6613 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:53:56.190799 6613 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:56.196630 6613 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.193:39249
I20260929 10:53:56.196667 6741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.193:39249 every 8 connection(s)
I20260929 10:53:56.197006 6613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
I20260929 10:53:56.201982 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 6613
I20260929 10:53:56.202086 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance
I20260929 10:53:56.202323 20803 external_mini_cluster.cc:1428] Setting key 22e2f68c009e164e5ca2d3cd9f2b6eaf
Sep 29 10:53:56 dist-test-slave-vqch krb5kdc[6505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679236, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:53:56.206959 6590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:56.197809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:42877 (local address 127.20.80.254:41547)
0929 10:53:56.197877 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:56.197879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:56.198069 (+ 190us) server_negotiation.cc:407] Connection header received
0929 10:53:56.198677 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:56.198678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:56.198688 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:56.198740 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:53:56.199766 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.200262 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.201342 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.201535 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.201589 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:56.204039 (+ 2450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:53:56.204041 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:53:56.204041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:53:56.204049 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:53:56.205537 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:56.205974 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:56.205975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:56.205976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:56.206037 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:56.206306 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:56.206307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:56.206308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:56.206422 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:53:56.206499 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:56.206776 (+ 277us) server_negotiation.cc:299] Negotiation successful
0929 10:53:56.206866 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 10:53:56.207165 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:56.197774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:41547 (local address 127.20.80.193:42877)
0929 10:53:56.197936 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:56.197979 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:56.198561 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:56.198820 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:56.198824 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:56.199475 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:53:56.199677 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:56.199681 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.200354 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.200355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:56.201263 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:56.201266 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.201640 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.201641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:56.201737 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:56.202361 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:53:56.202369 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:53:56.203939 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:53:56.205648 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:53:56.205656 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:53:56.205658 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:53:56.205880 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:53:56.206106 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:53:56.206107 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:53:56.206107 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:53:56.206230 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:53:56.206491 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:53:56.206498 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:53:56.206639 (+ 141us) client_negotiation.cc:769] Sending connection context
0929 10:53:56.206754 (+ 115us) client_negotiation.cc:240] Negotiation successful
0929 10:53:56.206910 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
I20260929 10:53:56.207672 6742 heartbeater.cc:347] Connected to a master server at 127.20.80.254:41547
I20260929 10:53:56.207762 6742 heartbeater.cc:464] Registering TS with master...
I20260929 10:53:56.207965 6742 heartbeater.cc:511] Master 127.20.80.254:41547 requested a full tablet report, sending...
I20260929 10:53:56.208546 6550 ts_manager.cc:194] Registered new tserver with Master: e4700fe2cf334f018612deaa06dff964 (127.20.80.193:39249)
I20260929 10:53:56.209218 6550 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.193@KRBTEST.COM'} at 127.20.80.193:42877
WARNING: no policy specified for kudu/127.20.80.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
I20260929 10:53:56.228389 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.194:0
--local_ip_for_outbound_sockets=127.20.80.194
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:41547
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:35891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:53:56.309255 6747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:53:56.309468 6747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:53:56.309490 6747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:53:56.310658 6747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:53:56.310699 6747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:53:56.310726 6747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.194
I20260929 10:53:56.312155 6747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:35891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:41547
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.194
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:53:56.312413 6747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:53:56.312651 6747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 10:53:56.318071 6747 server_base.cc:1034] running on GCE node
W20260929 10:53:56.318048 6753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:56.318045 6752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:56.318208 6755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:53:56.318423 6747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:53:56.318620 6747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:53:56.319772 6747 hybrid_clock.cc:648] HybridClock initialized: now 1790679236319749 us; error 40 us; skew 500 ppm
Sep 29 10:53:56 dist-test-slave-vqch krb5kdc[6505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679236, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:53:56.322405 6747 init.cc:377] Logged in from keytab as kudu/127.20.80.194@KRBTEST.COM (short username kudu)
I20260929 10:53:56.322906 6747 webserver.cc:460] Webserver started at http://127.20.80.194:35249/ using document root <none> and password file <none>
I20260929 10:53:56.323156 6747 fs_manager.cc:359] Metadata directory not provided
I20260929 10:53:56.323205 6747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:53:56.323352 6747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:53:56.324404 6747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-1/data/instance:
uuid: "98886686b62e4dcca2b950f7de325285"
format_stamp: "Formatted at 2026-09-29 10:53:56 on dist-test-slave-vqch"
server_key: "3c4eddcb7061148c479200bb4a66c579"
server_key_iv: "c925b7c17ea47744ce406f2545da5c36"
server_key_version: "encryptionkey@0"
I20260929 10:53:56.324770 6747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance:
uuid: "98886686b62e4dcca2b950f7de325285"
format_stamp: "Formatted at 2026-09-29 10:53:56 on dist-test-slave-vqch"
server_key: "3c4eddcb7061148c479200bb4a66c579"
server_key_iv: "c925b7c17ea47744ce406f2545da5c36"
server_key_version: "encryptionkey@0"
I20260929 10:53:56.326152 6747 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260929 10:53:56.327025 6762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:56.327232 6747 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:53:56.327335 6747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-1/wal
uuid: "98886686b62e4dcca2b950f7de325285"
format_stamp: "Formatted at 2026-09-29 10:53:56 on dist-test-slave-vqch"
server_key: "3c4eddcb7061148c479200bb4a66c579"
server_key_iv: "c925b7c17ea47744ce406f2545da5c36"
server_key_version: "encryptionkey@0"
I20260929 10:53:56.327422 6747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:53:56.337237 6747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:53:56.339205 6747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:53:56.339378 6747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:53:56.339596 6747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:53:56.339905 6747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:53:56.339943 6747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:56.339965 6747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:53:56.340010 6747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:56.345928 6747 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.194:43275
I20260929 10:53:56.345978 6877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.194:43275 every 8 connection(s)
I20260929 10:53:56.346280 6747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
Sep 29 10:53:56 dist-test-slave-vqch krb5kdc[6505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679236, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:53:56.353564 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 6747
I20260929 10:53:56.353744 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance
I20260929 10:53:56.353998 20803 external_mini_cluster.cc:1428] Setting key 1664f7e15a4b3ea66db82a91604cef53
I20260929 10:53:56.356077 6590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:56.346953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:58951 (local address 127.20.80.254:41547)
0929 10:53:56.347023 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:56.347026 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:56.347230 (+ 204us) server_negotiation.cc:407] Connection header received
0929 10:53:56.347747 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:56.347748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:56.347760 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:56.347829 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:53:56.348789 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.349399 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.350416 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.350605 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.350697 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:56.352877 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:53:56.352879 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:53:56.352879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:53:56.352887 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:53:56.354567 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:56.355030 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:56.355031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:56.355032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:56.355095 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:56.355391 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:56.355392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:56.355392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:56.355521 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:53:56.355622 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:56.355897 (+ 275us) server_negotiation.cc:299] Negotiation successful
0929 10:53:56.355985 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 10:53:56.356648 6880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:56.346910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:41547 (local address 127.20.80.194:58951)
0929 10:53:56.347087 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:56.347128 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:56.347648 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:56.347899 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:56.347902 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:56.348474 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:53:56.348665 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:56.348670 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.349501 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.349502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:56.350323 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:56.350325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.350718 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.350719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:56.350821 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:56.351352 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:53:56.351360 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:53:56.352771 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:53:56.354695 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:53:56.354703 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:53:56.354706 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:53:56.354935 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:53:56.355170 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:53:56.355171 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:53:56.355171 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:53:56.355310 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:53:56.355609 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:53:56.355616 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:53:56.355762 (+ 146us) client_negotiation.cc:769] Sending connection context
0929 10:53:56.355876 (+ 114us) client_negotiation.cc:240] Negotiation successful
0929 10:53:56.356041 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":106,"threads_started":1}
I20260929 10:53:56.357142 6878 heartbeater.cc:347] Connected to a master server at 127.20.80.254:41547
I20260929 10:53:56.357225 6878 heartbeater.cc:464] Registering TS with master...
I20260929 10:53:56.357425 6878 heartbeater.cc:511] Master 127.20.80.254:41547 requested a full tablet report, sending...
I20260929 10:53:56.357847 6550 ts_manager.cc:194] Registered new tserver with Master: 98886686b62e4dcca2b950f7de325285 (127.20.80.194:43275)
I20260929 10:53:56.358201 6550 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.194@KRBTEST.COM'} at 127.20.80.194:58951
WARNING: no policy specified for kudu/127.20.80.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
I20260929 10:53:56.380384 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.195:0
--local_ip_for_outbound_sockets=127.20.80.195
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:41547
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:35891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:53:56.463725 6883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:53:56.463966 6883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:53:56.464003 6883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:53:56.465161 6883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:53:56.465226 6883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:53:56.465270 6883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.195
I20260929 10:53:56.466724 6883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:35891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:41547
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.195
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:53:56.466991 6883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:53:56.467227 6883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:53:56.472678 6888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:56.473034 6891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:56.473044 6889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:53:56.473066 6883 server_base.cc:1034] running on GCE node
I20260929 10:53:56.473397 6883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:53:56.473671 6883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:53:56.474905 6883 hybrid_clock.cc:648] HybridClock initialized: now 1790679236474903 us; error 45 us; skew 500 ppm
Sep 29 10:53:56 dist-test-slave-vqch krb5kdc[6505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679236, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:53:56.477640 6883 init.cc:377] Logged in from keytab as kudu/127.20.80.195@KRBTEST.COM (short username kudu)
I20260929 10:53:56.478128 6883 webserver.cc:460] Webserver started at http://127.20.80.195:39897/ using document root <none> and password file <none>
I20260929 10:53:56.478399 6883 fs_manager.cc:359] Metadata directory not provided
I20260929 10:53:56.478469 6883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:53:56.478613 6883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:53:56.479720 6883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-2/data/instance:
uuid: "0ca1510ccaf64725aa8f2c6b39c71fec"
format_stamp: "Formatted at 2026-09-29 10:53:56 on dist-test-slave-vqch"
server_key: "34e115a7ac3f4d127832be4bdebeb64c"
server_key_iv: "323d94938a4eee70c42221cc97f355d3"
server_key_version: "encryptionkey@0"
I20260929 10:53:56.480104 6883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance:
uuid: "0ca1510ccaf64725aa8f2c6b39c71fec"
format_stamp: "Formatted at 2026-09-29 10:53:56 on dist-test-slave-vqch"
server_key: "34e115a7ac3f4d127832be4bdebeb64c"
server_key_iv: "323d94938a4eee70c42221cc97f355d3"
server_key_version: "encryptionkey@0"
I20260929 10:53:56.481547 6883 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 10:53:56.482326 6898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:56.482549 6883 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:53:56.482654 6883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-2/wal
uuid: "0ca1510ccaf64725aa8f2c6b39c71fec"
format_stamp: "Formatted at 2026-09-29 10:53:56 on dist-test-slave-vqch"
server_key: "34e115a7ac3f4d127832be4bdebeb64c"
server_key_iv: "323d94938a4eee70c42221cc97f355d3"
server_key_version: "encryptionkey@0"
I20260929 10:53:56.482739 6883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:53:56.522250 6883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:53:56.524688 6883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:53:56.524860 6883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:53:56.525159 6883 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:53:56.525488 6883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:53:56.525550 6883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:56.525586 6883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:53:56.525614 6883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:56.531666 6883 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.195:35709
I20260929 10:53:56.531726 7012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.195:35709 every 8 connection(s)
I20260929 10:53:56.532033 6883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
I20260929 10:53:56.536002 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 6883
I20260929 10:53:56.536123 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance
I20260929 10:53:56.536283 20803 external_mini_cluster.cc:1428] Setting key 1ecb3f8d8615673852189461f4949c66
Sep 29 10:53:56 dist-test-slave-vqch krb5kdc[6505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679236, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:53:56.541460 6590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:56.532764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:42241 (local address 127.20.80.254:41547)
0929 10:53:56.532843 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:56.532846 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:56.533014 (+ 168us) server_negotiation.cc:407] Connection header received
0929 10:53:56.533485 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:56.533486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:56.533496 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:56.533549 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:53:56.534441 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.534943 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.535944 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.536213 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.536280 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:56.538627 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:53:56.538628 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:53:56.538629 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:53:56.538637 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:53:56.539984 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:56.540399 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:56.540400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:56.540400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:56.540458 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:56.540780 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:56.540781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:56.540781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:56.540905 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:53:56.540976 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:56.541290 (+ 314us) server_negotiation.cc:299] Negotiation successful
0929 10:53:56.541371 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 10:53:56.541646 7016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:56.532717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:41547 (local address 127.20.80.195:42241)
0929 10:53:56.532883 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:56.532918 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:56.533403 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:56.533601 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:56.533605 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:56.534157 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:53:56.534358 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:56.534363 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.535028 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.535028 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:56.535844 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:56.535846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.536286 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.536287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:56.536433 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:56.537047 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:53:56.537057 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:53:56.538514 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:53:56.540072 (+ 1558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:53:56.540079 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:53:56.540082 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:53:56.540309 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:53:56.540579 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:53:56.540580 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:53:56.540581 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:53:56.540703 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:53:56.541005 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:53:56.541012 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:53:56.541148 (+ 136us) client_negotiation.cc:769] Sending connection context
0929 10:53:56.541257 (+ 109us) client_negotiation.cc:240] Negotiation successful
0929 10:53:56.541415 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":105,"threads_started":1}
I20260929 10:53:56.542142 7013 heartbeater.cc:347] Connected to a master server at 127.20.80.254:41547
I20260929 10:53:56.542224 7013 heartbeater.cc:464] Registering TS with master...
I20260929 10:53:56.542371 7013 heartbeater.cc:511] Master 127.20.80.254:41547 requested a full tablet report, sending...
I20260929 10:53:56.542706 6550 ts_manager.cc:194] Registered new tserver with Master: 0ca1510ccaf64725aa8f2c6b39c71fec (127.20.80.195:35709)
I20260929 10:53:56.543030 6550 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.195@KRBTEST.COM'} at 127.20.80.195:42241
I20260929 10:53:56.549089 20803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:53:56.556723 6590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:56.550424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38768 (local address 127.20.80.254:41547)
0929 10:53:56.550461 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:56.550463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:56.550571 (+ 108us) server_negotiation.cc:407] Connection header received
0929 10:53:56.550622 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:56.550622 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:56.550631 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:56.550682 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:53:56.551197 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.551678 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.552382 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.552563 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.552616 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:56.554124 (+ 1508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:53:56.554125 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:53:56.554125 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:53:56.554131 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:53:56.555561 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:56.556001 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:56.556001 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:56.556002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:56.556052 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:56.556252 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:56.556253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:56.556253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:56.556381 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:53:56.556461 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:56.556556 (+ 95us) server_negotiation.cc:299] Negotiation successful
0929 10:53:56.556604 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 10:53:56.558394 6550 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38768:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:53:56.559937 6550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:53:56.566439 7025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:56.562316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:43275 (local address 127.0.0.1:43232)
0929 10:53:56.562489 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:56.562531 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:56.562601 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:56.562844 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:56.562847 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:56.562850 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:53:56.562928 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:56.562929 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.564016 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.564017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:56.565230 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:56.565233 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.566115 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.566116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:56.566204 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:56.566216 (+ 12us) client_negotiation.cc:769] Sending connection context
0929 10:53:56.566311 (+ 95us) client_negotiation.cc:240] Negotiation successful
0929 10:53:56.566365 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":68,"threads_started":1}
I20260929 10:53:56.566452 7028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:56.562423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43232 (local address 127.20.80.194:43275)
0929 10:53:56.562638 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:56.562644 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:56.562666 (+ 22us) server_negotiation.cc:407] Connection header received
0929 10:53:56.562705 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:56.562707 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:56.562725 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:56.562828 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:53:56.563087 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.563913 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.565304 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.566022 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.566104 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:56.566164 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:56.566328 (+ 164us) server_negotiation.cc:299] Negotiation successful
0929 10:53:56.566369 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":91,"threads_started":1}
I20260929 10:53:56.566926 7027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:56.562332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:39249 (local address 127.0.0.1:44608)
0929 10:53:56.562937 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:56.562947 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:56.562989 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:56.563303 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:56.563304 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:56.563307 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:53:56.563351 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:56.563352 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.564616 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.564617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:56.565759 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:56.565762 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.566680 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.566681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:56.566746 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:56.566750 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:53:56.566803 (+ 53us) client_negotiation.cc:240] Negotiation successful
0929 10:53:56.566846 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"mutex_wait_us":38,"thread_start_us":95,"threads_started":1}
I20260929 10:53:56.567101 7026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:56.562330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:35709 (local address 127.0.0.1:42998)
0929 10:53:56.562769 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:56.562780 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:56.562844 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:56.563267 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:56.563268 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:56.563270 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:53:56.563316 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:56.563317 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.564596 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.564597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:56.565631 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:56.565633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.566627 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.566628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:56.566819 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:56.566821 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 10:53:56.566863 (+ 42us) client_negotiation.cc:240] Negotiation successful
0929 10:53:56.566894 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"mutex_wait_us":54,"thread_start_us":97,"threads_started":1}
I20260929 10:53:56.567122 7030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:56.562572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42998 (local address 127.20.80.195:35709)
0929 10:53:56.562791 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:56.562796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:56.563009 (+ 213us) server_negotiation.cc:407] Connection header received
0929 10:53:56.563054 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:56.563056 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:56.563073 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:56.563196 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:53:56.563381 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.564482 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.565713 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.566458 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.566531 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:56.566590 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:56.566971 (+ 381us) server_negotiation.cc:299] Negotiation successful
0929 10:53:56.567019 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":133,"threads_started":1}
I20260929 10:53:56.567399 6810 tablet_service.cc:1467] Processing CreateTablet for tablet e4f238ee8f974ccea5d3d37663cbbcb0 (DEFAULT_TABLE table=test-table [id=e6c6efbf03b04b158432e825fd933191]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:53:56.567761 6810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4f238ee8f974ccea5d3d37663cbbcb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:53:56.567749 6947 tablet_service.cc:1467] Processing CreateTablet for tablet e4f238ee8f974ccea5d3d37663cbbcb0 (DEFAULT_TABLE table=test-table [id=e6c6efbf03b04b158432e825fd933191]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:53:56.568074 6947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4f238ee8f974ccea5d3d37663cbbcb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:53:56.569171 7029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:56.562592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44608 (local address 127.20.80.193:39249)
0929 10:53:56.562828 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:56.562832 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:56.563062 (+ 230us) server_negotiation.cc:407] Connection header received
0929 10:53:56.563110 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:56.563112 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:56.563138 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:56.563213 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:53:56.563432 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.564517 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.565848 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.566531 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.566589 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:56.566648 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:56.566824 (+ 176us) server_negotiation.cc:299] Negotiation successful
0929 10:53:56.566879 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":77,"threads_started":1}
I20260929 10:53:56.570686 7031 tablet_bootstrap.cc:492] T e4f238ee8f974ccea5d3d37663cbbcb0 P 98886686b62e4dcca2b950f7de325285: Bootstrap starting.
I20260929 10:53:56.570984 6676 tablet_service.cc:1467] Processing CreateTablet for tablet e4f238ee8f974ccea5d3d37663cbbcb0 (DEFAULT_TABLE table=test-table [id=e6c6efbf03b04b158432e825fd933191]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:53:56.571197 7032 tablet_bootstrap.cc:492] T e4f238ee8f974ccea5d3d37663cbbcb0 P 0ca1510ccaf64725aa8f2c6b39c71fec: Bootstrap starting.
I20260929 10:53:56.571305 6676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4f238ee8f974ccea5d3d37663cbbcb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:53:56.571410 7031 tablet_bootstrap.cc:654] T e4f238ee8f974ccea5d3d37663cbbcb0 P 98886686b62e4dcca2b950f7de325285: Neither blocks nor log segments found. Creating new log.
I20260929 10:53:56.571892 7032 tablet_bootstrap.cc:654] T e4f238ee8f974ccea5d3d37663cbbcb0 P 0ca1510ccaf64725aa8f2c6b39c71fec: Neither blocks nor log segments found. Creating new log.
I20260929 10:53:56.572261 7031 log.cc:824] T e4f238ee8f974ccea5d3d37663cbbcb0 P 98886686b62e4dcca2b950f7de325285: Log is configured to *not* fsync() on all Append() calls
I20260929 10:53:56.572455 7032 log.cc:824] T e4f238ee8f974ccea5d3d37663cbbcb0 P 0ca1510ccaf64725aa8f2c6b39c71fec: Log is configured to *not* fsync() on all Append() calls
I20260929 10:53:56.573133 7031 tablet_bootstrap.cc:492] T e4f238ee8f974ccea5d3d37663cbbcb0 P 98886686b62e4dcca2b950f7de325285: No bootstrap required, opened a new log
I20260929 10:53:56.573200 7032 tablet_bootstrap.cc:492] T e4f238ee8f974ccea5d3d37663cbbcb0 P 0ca1510ccaf64725aa8f2c6b39c71fec: No bootstrap required, opened a new log
I20260929 10:53:56.573208 7031 ts_tablet_manager.cc:1397] T e4f238ee8f974ccea5d3d37663cbbcb0 P 98886686b62e4dcca2b950f7de325285: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 10:53:56.573261 7032 ts_tablet_manager.cc:1397] T e4f238ee8f974ccea5d3d37663cbbcb0 P 0ca1510ccaf64725aa8f2c6b39c71fec: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:53:56.574295 7035 tablet_bootstrap.cc:492] T e4f238ee8f974ccea5d3d37663cbbcb0 P e4700fe2cf334f018612deaa06dff964: Bootstrap starting.
I20260929 10:53:56.574847 7035 tablet_bootstrap.cc:654] T e4f238ee8f974ccea5d3d37663cbbcb0 P e4700fe2cf334f018612deaa06dff964: Neither blocks nor log segments found. Creating new log.
I20260929 10:53:56.575161 7032 raft_consensus.cc:348] T e4f238ee8f974ccea5d3d37663cbbcb0 P 0ca1510ccaf64725aa8f2c6b39c71fec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1510ccaf64725aa8f2c6b39c71fec" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35709 } } peers { permanent_uuid: "98886686b62e4dcca2b950f7de325285" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 43275 } } peers { permanent_uuid: "e4700fe2cf334f018612deaa06dff964" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 39249 } }
I20260929 10:53:56.575186 7031 raft_consensus.cc:348] T e4f238ee8f974ccea5d3d37663cbbcb0 P 98886686b62e4dcca2b950f7de325285 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1510ccaf64725aa8f2c6b39c71fec" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35709 } } peers { permanent_uuid: "98886686b62e4dcca2b950f7de325285" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 43275 } } peers { permanent_uuid: "e4700fe2cf334f018612deaa06dff964" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 39249 } }
I20260929 10:53:56.575356 7032 raft_consensus.cc:374] T e4f238ee8f974ccea5d3d37663cbbcb0 P 0ca1510ccaf64725aa8f2c6b39c71fec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:53:56.575356 7031 raft_consensus.cc:374] T e4f238ee8f974ccea5d3d37663cbbcb0 P 98886686b62e4dcca2b950f7de325285 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:53:56.575394 7032 raft_consensus.cc:729] T e4f238ee8f974ccea5d3d37663cbbcb0 P 0ca1510ccaf64725aa8f2c6b39c71fec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ca1510ccaf64725aa8f2c6b39c71fec, State: Initialized, Role: FOLLOWER
I20260929 10:53:56.575394 7031 raft_consensus.cc:729] T e4f238ee8f974ccea5d3d37663cbbcb0 P 98886686b62e4dcca2b950f7de325285 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98886686b62e4dcca2b950f7de325285, State: Initialized, Role: FOLLOWER
I20260929 10:53:56.575381 7035 log.cc:824] T e4f238ee8f974ccea5d3d37663cbbcb0 P e4700fe2cf334f018612deaa06dff964: Log is configured to *not* fsync() on all Append() calls
I20260929 10:53:56.575524 7032 consensus_queue.cc:260] T e4f238ee8f974ccea5d3d37663cbbcb0 P 0ca1510ccaf64725aa8f2c6b39c71fec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1510ccaf64725aa8f2c6b39c71fec" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35709 } } peers { permanent_uuid: "98886686b62e4dcca2b950f7de325285" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 43275 } } peers { permanent_uuid: "e4700fe2cf334f018612deaa06dff964" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 39249 } }
I20260929 10:53:56.575513 7031 consensus_queue.cc:260] T e4f238ee8f974ccea5d3d37663cbbcb0 P 98886686b62e4dcca2b950f7de325285 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1510ccaf64725aa8f2c6b39c71fec" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35709 } } peers { permanent_uuid: "98886686b62e4dcca2b950f7de325285" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 43275 } } peers { permanent_uuid: "e4700fe2cf334f018612deaa06dff964" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 39249 } }
I20260929 10:53:56.575781 7031 ts_tablet_manager.cc:1428] T e4f238ee8f974ccea5d3d37663cbbcb0 P 98886686b62e4dcca2b950f7de325285: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 10:53:56.575793 7032 ts_tablet_manager.cc:1428] T e4f238ee8f974ccea5d3d37663cbbcb0 P 0ca1510ccaf64725aa8f2c6b39c71fec: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 10:53:56.575920 6878 heartbeater.cc:503] Master 127.20.80.254:41547 was elected leader, sending a full tablet report...
I20260929 10:53:56.575919 7013 heartbeater.cc:503] Master 127.20.80.254:41547 was elected leader, sending a full tablet report...
I20260929 10:53:56.576453 7035 tablet_bootstrap.cc:492] T e4f238ee8f974ccea5d3d37663cbbcb0 P e4700fe2cf334f018612deaa06dff964: No bootstrap required, opened a new log
I20260929 10:53:56.576524 7035 ts_tablet_manager.cc:1397] T e4f238ee8f974ccea5d3d37663cbbcb0 P e4700fe2cf334f018612deaa06dff964: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:53:56.578125 7035 raft_consensus.cc:348] T e4f238ee8f974ccea5d3d37663cbbcb0 P e4700fe2cf334f018612deaa06dff964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1510ccaf64725aa8f2c6b39c71fec" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35709 } } peers { permanent_uuid: "98886686b62e4dcca2b950f7de325285" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 43275 } } peers { permanent_uuid: "e4700fe2cf334f018612deaa06dff964" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 39249 } }
I20260929 10:53:56.578255 7035 raft_consensus.cc:374] T e4f238ee8f974ccea5d3d37663cbbcb0 P e4700fe2cf334f018612deaa06dff964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:53:56.578279 7035 raft_consensus.cc:729] T e4f238ee8f974ccea5d3d37663cbbcb0 P e4700fe2cf334f018612deaa06dff964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4700fe2cf334f018612deaa06dff964, State: Initialized, Role: FOLLOWER
I20260929 10:53:56.578419 7035 consensus_queue.cc:260] T e4f238ee8f974ccea5d3d37663cbbcb0 P e4700fe2cf334f018612deaa06dff964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1510ccaf64725aa8f2c6b39c71fec" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35709 } } peers { permanent_uuid: "98886686b62e4dcca2b950f7de325285" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 43275 } } peers { permanent_uuid: "e4700fe2cf334f018612deaa06dff964" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 39249 } }
I20260929 10:53:56.578676 7035 ts_tablet_manager.cc:1428] T e4f238ee8f974ccea5d3d37663cbbcb0 P e4700fe2cf334f018612deaa06dff964: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:53:56.578851 6742 heartbeater.cc:503] Master 127.20.80.254:41547 was elected leader, sending a full tablet report...
W20260929 10:53:56.596899 6879 tablet.cc:2369] T e4f238ee8f974ccea5d3d37663cbbcb0 P 98886686b62e4dcca2b950f7de325285: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:53:56.697768 6743 tablet.cc:2369] T e4f238ee8f974ccea5d3d37663cbbcb0 P e4700fe2cf334f018612deaa06dff964: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:53:56.771354 7039 raft_consensus.cc:482] T e4f238ee8f974ccea5d3d37663cbbcb0 P e4700fe2cf334f018612deaa06dff964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:53:56.771752 7039 raft_consensus.cc:504] T e4f238ee8f974ccea5d3d37663cbbcb0 P e4700fe2cf334f018612deaa06dff964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1510ccaf64725aa8f2c6b39c71fec" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35709 } } peers { permanent_uuid: "98886686b62e4dcca2b950f7de325285" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 43275 } } peers { permanent_uuid: "e4700fe2cf334f018612deaa06dff964" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 39249 } }
I20260929 10:53:56.772197 7039 leader_election.cc:288] T e4f238ee8f974ccea5d3d37663cbbcb0 P e4700fe2cf334f018612deaa06dff964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ca1510ccaf64725aa8f2c6b39c71fec (127.20.80.195:35709), 98886686b62e4dcca2b950f7de325285 (127.20.80.194:43275)
I20260929 10:53:56.775256 7030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:56.772524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:56663 (local address 127.20.80.195:35709)
0929 10:53:56.772569 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:56.772571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:56.772672 (+ 101us) server_negotiation.cc:407] Connection header received
0929 10:53:56.772708 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:56.772710 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:56.772720 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:56.772779 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:53:56.772997 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.773508 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.774499 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.774892 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.774947 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:56.774955 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:56.775140 (+ 185us) server_negotiation.cc:299] Negotiation successful
0929 10:53:56.775185 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 10:53:56.775267 7041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:56.772414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:35709 (local address 127.20.80.193:56663)
0929 10:53:56.772595 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:56.772605 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:56.772663 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:56.772809 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:56.772809 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:56.772811 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:53:56.772894 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:56.772897 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.773589 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.773589 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:56.774402 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:56.774404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.774982 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.774984 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:56.775055 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:56.775059 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:53:56.775110 (+ 51us) client_negotiation.cc:240] Negotiation successful
0929 10:53:56.775141 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":87,"threads_started":1}
I20260929 10:53:56.775319 7028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:56.772489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:33013 (local address 127.20.80.194:43275)
0929 10:53:56.772540 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:56.772543 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:56.772769 (+ 226us) server_negotiation.cc:407] Connection header received
0929 10:53:56.772808 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:56.772809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:56.772823 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:56.772875 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:53:56.773095 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.773535 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.774534 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.774904 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.774999 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:56.775008 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:56.775196 (+ 188us) server_negotiation.cc:299] Negotiation successful
0929 10:53:56.775241 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260929 10:53:56.775286 7040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:56.772414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:43275 (local address 127.20.80.193:33013)
0929 10:53:56.772612 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:56.772623 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:56.772714 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:56.772885 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:56.772886 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:56.772888 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:53:56.773015 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:56.773017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.773644 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.773645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:56.774407 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:56.774409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.775032 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.775033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:56.775085 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:56.775088 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:53:56.775170 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 10:53:56.775206 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":3,"thread_start_us":91,"threads_started":1}
I20260929 10:53:56.775709 6967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4f238ee8f974ccea5d3d37663cbbcb0" candidate_uuid: "e4700fe2cf334f018612deaa06dff964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ca1510ccaf64725aa8f2c6b39c71fec" is_pre_election: true
I20260929 10:53:56.775872 6967 raft_consensus.cc:2454] T e4f238ee8f974ccea5d3d37663cbbcb0 P 0ca1510ccaf64725aa8f2c6b39c71fec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4700fe2cf334f018612deaa06dff964 in term 0.
I20260929 10:53:56.775836 6830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4f238ee8f974ccea5d3d37663cbbcb0" candidate_uuid: "e4700fe2cf334f018612deaa06dff964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98886686b62e4dcca2b950f7de325285" is_pre_election: true
I20260929 10:53:56.775961 6830 raft_consensus.cc:2454] T e4f238ee8f974ccea5d3d37663cbbcb0 P 98886686b62e4dcca2b950f7de325285 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4700fe2cf334f018612deaa06dff964 in term 0.
I20260929 10:53:56.776090 6632 leader_election.cc:302] T e4f238ee8f974ccea5d3d37663cbbcb0 P e4700fe2cf334f018612deaa06dff964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ca1510ccaf64725aa8f2c6b39c71fec, e4700fe2cf334f018612deaa06dff964; no voters:
I20260929 10:53:56.776232 7039 raft_consensus.cc:2781] T e4f238ee8f974ccea5d3d37663cbbcb0 P e4700fe2cf334f018612deaa06dff964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:53:56.776289 7039 raft_consensus.cc:482] T e4f238ee8f974ccea5d3d37663cbbcb0 P e4700fe2cf334f018612deaa06dff964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:53:56.776315 7039 raft_consensus.cc:3037] T e4f238ee8f974ccea5d3d37663cbbcb0 P e4700fe2cf334f018612deaa06dff964 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:53:56.777179 7039 raft_consensus.cc:504] T e4f238ee8f974ccea5d3d37663cbbcb0 P e4700fe2cf334f018612deaa06dff964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1510ccaf64725aa8f2c6b39c71fec" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35709 } } peers { permanent_uuid: "98886686b62e4dcca2b950f7de325285" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 43275 } } peers { permanent_uuid: "e4700fe2cf334f018612deaa06dff964" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 39249 } }
I20260929 10:53:56.777328 7039 leader_election.cc:288] T e4f238ee8f974ccea5d3d37663cbbcb0 P e4700fe2cf334f018612deaa06dff964 [CANDIDATE]: Term 1 election: Requested vote from peers 0ca1510ccaf64725aa8f2c6b39c71fec (127.20.80.195:35709), 98886686b62e4dcca2b950f7de325285 (127.20.80.194:43275)
I20260929 10:53:56.777576 6830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4f238ee8f974ccea5d3d37663cbbcb0" candidate_uuid: "e4700fe2cf334f018612deaa06dff964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98886686b62e4dcca2b950f7de325285"
I20260929 10:53:56.777576 6967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4f238ee8f974ccea5d3d37663cbbcb0" candidate_uuid: "e4700fe2cf334f018612deaa06dff964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ca1510ccaf64725aa8f2c6b39c71fec"
I20260929 10:53:56.777664 6830 raft_consensus.cc:3037] T e4f238ee8f974ccea5d3d37663cbbcb0 P 98886686b62e4dcca2b950f7de325285 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:53:56.777681 6967 raft_consensus.cc:3037] T e4f238ee8f974ccea5d3d37663cbbcb0 P 0ca1510ccaf64725aa8f2c6b39c71fec [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:53:56.778594 6830 raft_consensus.cc:2454] T e4f238ee8f974ccea5d3d37663cbbcb0 P 98886686b62e4dcca2b950f7de325285 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4700fe2cf334f018612deaa06dff964 in term 1.
I20260929 10:53:56.778594 6967 raft_consensus.cc:2454] T e4f238ee8f974ccea5d3d37663cbbcb0 P 0ca1510ccaf64725aa8f2c6b39c71fec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4700fe2cf334f018612deaa06dff964 in term 1.
I20260929 10:53:56.778812 6631 leader_election.cc:302] T e4f238ee8f974ccea5d3d37663cbbcb0 P e4700fe2cf334f018612deaa06dff964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98886686b62e4dcca2b950f7de325285, e4700fe2cf334f018612deaa06dff964; no voters:
I20260929 10:53:56.778903 7039 raft_consensus.cc:2781] T e4f238ee8f974ccea5d3d37663cbbcb0 P e4700fe2cf334f018612deaa06dff964 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:53:56.779062 7039 raft_consensus.cc:686] T e4f238ee8f974ccea5d3d37663cbbcb0 P e4700fe2cf334f018612deaa06dff964 [term 1 LEADER]: Becoming Leader. State: Replica: e4700fe2cf334f018612deaa06dff964, State: Running, Role: LEADER
I20260929 10:53:56.779186 7039 consensus_queue.cc:237] T e4f238ee8f974ccea5d3d37663cbbcb0 P e4700fe2cf334f018612deaa06dff964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1510ccaf64725aa8f2c6b39c71fec" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35709 } } peers { permanent_uuid: "98886686b62e4dcca2b950f7de325285" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 43275 } } peers { permanent_uuid: "e4700fe2cf334f018612deaa06dff964" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 39249 } }
I20260929 10:53:56.780049 6550 catalog_manager.cc:5495] T e4f238ee8f974ccea5d3d37663cbbcb0 P e4700fe2cf334f018612deaa06dff964 reported cstate change: term changed from 0 to 1, leader changed from <none> to e4700fe2cf334f018612deaa06dff964 (127.20.80.193). New cstate: current_term: 1 leader_uuid: "e4700fe2cf334f018612deaa06dff964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1510ccaf64725aa8f2c6b39c71fec" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98886686b62e4dcca2b950f7de325285" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 43275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4700fe2cf334f018612deaa06dff964" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 39249 } health_report { overall_health: HEALTHY } } }
W20260929 10:53:56.782593 7014 tablet.cc:2369] T e4f238ee8f974ccea5d3d37663cbbcb0 P 0ca1510ccaf64725aa8f2c6b39c71fec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:53:56.795464 7029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:56.793049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44616 (local address 127.20.80.193:39249)
0929 10:53:56.793110 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:56.793112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:56.793120 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:53:56.793172 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:56.793173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:56.793196 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:56.793248 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:53:56.793418 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.793907 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.794656 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.794861 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.794938 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:56.795089 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:53:56.795191 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:53:56.795272 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:56.795324 (+ 52us) server_negotiation.cc:299] Negotiation successful
0929 10:53:56.795365 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 10:53:56.797482 6967 raft_consensus.cc:1261] T e4f238ee8f974ccea5d3d37663cbbcb0 P 0ca1510ccaf64725aa8f2c6b39c71fec [term 1 FOLLOWER]: Refusing update from remote peer e4700fe2cf334f018612deaa06dff964: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:53:56.797489 6830 raft_consensus.cc:1261] T e4f238ee8f974ccea5d3d37663cbbcb0 P 98886686b62e4dcca2b950f7de325285 [term 1 FOLLOWER]: Refusing update from remote peer e4700fe2cf334f018612deaa06dff964: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:53:56.797945 7042 consensus_queue.cc:1035] T e4f238ee8f974ccea5d3d37663cbbcb0 P e4700fe2cf334f018612deaa06dff964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98886686b62e4dcca2b950f7de325285" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 43275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:53:56.798241 7039 consensus_queue.cc:1035] T e4f238ee8f974ccea5d3d37663cbbcb0 P e4700fe2cf334f018612deaa06dff964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ca1510ccaf64725aa8f2c6b39c71fec" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:53:56.800787 7047 mvcc.cc:205] Tried to move back new op lower bound from 7334622153919582208 to 7334622153848295424. Current Snapshot: MvccSnapshot[applied={T|T < 7334622153919582208}]
I20260929 10:53:56.800793 7046 mvcc.cc:205] Tried to move back new op lower bound from 7334622153919582208 to 7334622153848295424. Current Snapshot: MvccSnapshot[applied={T|T < 7334622153919582208}]
I20260929 10:53:56.800966 7045 mvcc.cc:205] Tried to move back new op lower bound from 7334622153919582208 to 7334622153848295424. Current Snapshot: MvccSnapshot[applied={T|T < 7334622153919582208}]
I20260929 10:53:56.803467 6550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:53:56.803584 6550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:53:56.804539 6550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b3b8759eb7fe4af297a0d2061bfd8592: Sending DeleteTablet for 3 replicas of tablet e4f238ee8f974ccea5d3d37663cbbcb0
I20260929 10:53:56.804965 6676 tablet_service.cc:1514] Processing DeleteTablet for tablet e4f238ee8f974ccea5d3d37663cbbcb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:53:56 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:44608
I20260929 10:53:56.805012 6810 tablet_service.cc:1514] Processing DeleteTablet for tablet e4f238ee8f974ccea5d3d37663cbbcb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:53:56 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:43232
I20260929 10:53:56.805027 6947 tablet_service.cc:1514] Processing DeleteTablet for tablet e4f238ee8f974ccea5d3d37663cbbcb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:53:56 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:42998
I20260929 10:53:56.805241 7057 tablet_replica.cc:330] T e4f238ee8f974ccea5d3d37663cbbcb0 P 0ca1510ccaf64725aa8f2c6b39c71fec: stopping tablet replica
I20260929 10:53:56.805487 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 6613
I20260929 10:53:56.805382 7057 raft_consensus.cc:2229] T e4f238ee8f974ccea5d3d37663cbbcb0 P 0ca1510ccaf64725aa8f2c6b39c71fec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:53:56.806735 7057 raft_consensus.cc:2258] T e4f238ee8f974ccea5d3d37663cbbcb0 P 0ca1510ccaf64725aa8f2c6b39c71fec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:53:56.807288 7055 tablet_replica.cc:330] T e4f238ee8f974ccea5d3d37663cbbcb0 P 98886686b62e4dcca2b950f7de325285: stopping tablet replica
I20260929 10:53:56.807279 7057 ts_tablet_manager.cc:1905] T e4f238ee8f974ccea5d3d37663cbbcb0 P 0ca1510ccaf64725aa8f2c6b39c71fec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:53:56.807396 7055 raft_consensus.cc:2229] T e4f238ee8f974ccea5d3d37663cbbcb0 P 98886686b62e4dcca2b950f7de325285 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:53:56.807509 7055 raft_consensus.cc:2258] T e4f238ee8f974ccea5d3d37663cbbcb0 P 98886686b62e4dcca2b950f7de325285 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:53:56.807931 7055 ts_tablet_manager.cc:1905] T e4f238ee8f974ccea5d3d37663cbbcb0 P 98886686b62e4dcca2b950f7de325285: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:53:56.809273 7057 ts_tablet_manager.cc:1918] T e4f238ee8f974ccea5d3d37663cbbcb0 P 0ca1510ccaf64725aa8f2c6b39c71fec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:53:56.809350 7057 log.cc:1196] T e4f238ee8f974ccea5d3d37663cbbcb0 P 0ca1510ccaf64725aa8f2c6b39c71fec: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-2/wal/wals/e4f238ee8f974ccea5d3d37663cbbcb0
I20260929 10:53:56.809630 7057 ts_tablet_manager.cc:1939] T e4f238ee8f974ccea5d3d37663cbbcb0 P 0ca1510ccaf64725aa8f2c6b39c71fec: Deleting consensus metadata
I20260929 10:53:56.809652 7055 ts_tablet_manager.cc:1918] T e4f238ee8f974ccea5d3d37663cbbcb0 P 98886686b62e4dcca2b950f7de325285: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:53:56.809710 7055 log.cc:1196] T e4f238ee8f974ccea5d3d37663cbbcb0 P 98886686b62e4dcca2b950f7de325285: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790678996126930-20803-0/minicluster-data/ts-1/wal/wals/e4f238ee8f974ccea5d3d37663cbbcb0
I20260929 10:53:56.810006 7055 ts_tablet_manager.cc:1939] T e4f238ee8f974ccea5d3d37663cbbcb0 P 98886686b62e4dcca2b950f7de325285: Deleting consensus metadata
I20260929 10:53:56.810329 6537 catalog_manager.cc:4841] TS 0ca1510ccaf64725aa8f2c6b39c71fec (127.20.80.195:35709): tablet e4f238ee8f974ccea5d3d37663cbbcb0 (table test-table [id=e6c6efbf03b04b158432e825fd933191]) successfully deleted
I20260929 10:53:56.810587 6536 catalog_manager.cc:4841] TS 98886686b62e4dcca2b950f7de325285 (127.20.80.194:43275): tablet e4f238ee8f974ccea5d3d37663cbbcb0 (table test-table [id=e6c6efbf03b04b158432e825fd933191]) successfully deleted
W20260929 10:53:56.812870 6534 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.80.193:39249 (error 108)
I20260929 10:53:56.813442 7026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:56.813315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:39249 (local address 127.0.0.1:44628)
0929 10:53:56.813378 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:56.813415 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.193:39249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260929 10:53:56.813526 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 6747
W20260929 10:53:56.813570 6534 catalog_manager.cc:4584] TS e4700fe2cf334f018612deaa06dff964 (127.20.80.193:39249): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e4f238ee8f974ccea5d3d37663cbbcb0: Network error: Client connection negotiation failed: client connection to 127.20.80.193:39249: connect: Connection refused (error 111)
I20260929 10:53:56.818234 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 6883
I20260929 10:53:56.822675 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 6518
2026-09-29T10:53:56Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2181 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-29T10:53:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:53:56Z Disabled control of system clock
I20260929 10:53:56.852174 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:37813
--webserver_interface=127.20.80.254
--webserver_port=0
--builtin_ntp_servers=127.20.80.212:33091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:37813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 10:53:56.934964 7062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:53:56.935194 7062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:53:56.935214 7062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:53:56.936324 7062 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:53:56.936357 7062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:53:56.936370 7062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:53:56.936383 7062 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:53:56.936394 7062 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:53:56.937695 7062 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:33091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:37813
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:37813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:53:56.937892 7062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:53:56.938125 7062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:53:56.943087 7067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:56.943153 7068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:56.943140 7070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:53:56.943363 7062 server_base.cc:1034] running on GCE node
I20260929 10:53:56.943604 7062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:53:56.943938 7062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:53:56.945109 7062 hybrid_clock.cc:648] HybridClock initialized: now 1790679236945099 us; error 52 us; skew 500 ppm
I20260929 10:53:56.946322 7062 webserver.cc:460] Webserver started at http://127.20.80.254:46181/ using document root <none> and password file <none>
I20260929 10:53:56.946578 7062 fs_manager.cc:359] Metadata directory not provided
I20260929 10:53:56.946689 7062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:53:56.946841 7062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:53:56.947894 7062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "b0ab1515f9b343bd9d5c01bf6a8ee22c"
format_stamp: "Formatted at 2026-09-29 10:53:56 on dist-test-slave-vqch"
server_key: "e0c8ec6aa82c3364507e96edc5e5c7eb"
server_key_iv: "c3f6ce90ef4fd025bd55c7aa1f5896e8"
server_key_version: "encryptionkey@0"
I20260929 10:53:56.948247 7062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "b0ab1515f9b343bd9d5c01bf6a8ee22c"
format_stamp: "Formatted at 2026-09-29 10:53:56 on dist-test-slave-vqch"
server_key: "e0c8ec6aa82c3364507e96edc5e5c7eb"
server_key_iv: "c3f6ce90ef4fd025bd55c7aa1f5896e8"
server_key_version: "encryptionkey@0"
I20260929 10:53:56.949580 7062 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:53:56.950330 7076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:56.950498 7062 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:53:56.950601 7062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "b0ab1515f9b343bd9d5c01bf6a8ee22c"
format_stamp: "Formatted at 2026-09-29 10:53:56 on dist-test-slave-vqch"
server_key: "e0c8ec6aa82c3364507e96edc5e5c7eb"
server_key_iv: "c3f6ce90ef4fd025bd55c7aa1f5896e8"
server_key_version: "encryptionkey@0"
I20260929 10:53:56.950680 7062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:53:56.975970 7062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:53:56.976313 7062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:53:56.976475 7062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:53:56.980583 7062 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:37813
I20260929 10:53:56.980623 7128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:37813 every 8 connection(s)
I20260929 10:53:56.981042 7062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:53:56.982082 7129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:53:56.984313 7129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0ab1515f9b343bd9d5c01bf6a8ee22c: Bootstrap starting.
I20260929 10:53:56.984939 7129 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0ab1515f9b343bd9d5c01bf6a8ee22c: Neither blocks nor log segments found. Creating new log.
I20260929 10:53:56.985276 7129 log.cc:824] T 00000000000000000000000000000000 P b0ab1515f9b343bd9d5c01bf6a8ee22c: Log is configured to *not* fsync() on all Append() calls
I20260929 10:53:56.985935 7129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0ab1515f9b343bd9d5c01bf6a8ee22c: No bootstrap required, opened a new log
I20260929 10:53:56.986929 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 7062
I20260929 10:53:56.987040 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:53:56.987257 20803 external_mini_cluster.cc:1428] Setting key cae2c6408206194e7a54bcc7efcfedc1
I20260929 10:53:56.988082 7129 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0ab1515f9b343bd9d5c01bf6a8ee22c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ab1515f9b343bd9d5c01bf6a8ee22c" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37813 } }
I20260929 10:53:56.988247 7129 raft_consensus.cc:374] T 00000000000000000000000000000000 P b0ab1515f9b343bd9d5c01bf6a8ee22c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:53:56.988308 7129 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0ab1515f9b343bd9d5c01bf6a8ee22c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0ab1515f9b343bd9d5c01bf6a8ee22c, State: Initialized, Role: FOLLOWER
I20260929 10:53:56.988462 7129 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0ab1515f9b343bd9d5c01bf6a8ee22c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ab1515f9b343bd9d5c01bf6a8ee22c" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37813 } }
I20260929 10:53:56.988569 7129 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0ab1515f9b343bd9d5c01bf6a8ee22c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:53:56.988615 7129 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0ab1515f9b343bd9d5c01bf6a8ee22c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:53:56.988664 7129 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0ab1515f9b343bd9d5c01bf6a8ee22c [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:53:56.989269 7129 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0ab1515f9b343bd9d5c01bf6a8ee22c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ab1515f9b343bd9d5c01bf6a8ee22c" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37813 } }
I20260929 10:53:56.989383 7129 leader_election.cc:302] T 00000000000000000000000000000000 P b0ab1515f9b343bd9d5c01bf6a8ee22c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0ab1515f9b343bd9d5c01bf6a8ee22c; no voters:
I20260929 10:53:56.989576 7129 leader_election.cc:288] T 00000000000000000000000000000000 P b0ab1515f9b343bd9d5c01bf6a8ee22c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:53:56.989607 7134 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0ab1515f9b343bd9d5c01bf6a8ee22c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:53:56.989717 7134 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0ab1515f9b343bd9d5c01bf6a8ee22c [term 1 LEADER]: Becoming Leader. State: Replica: b0ab1515f9b343bd9d5c01bf6a8ee22c, State: Running, Role: LEADER
I20260929 10:53:56.989847 7129 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0ab1515f9b343bd9d5c01bf6a8ee22c [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:53:56.989854 7134 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0ab1515f9b343bd9d5c01bf6a8ee22c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ab1515f9b343bd9d5c01bf6a8ee22c" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37813 } }
I20260929 10:53:56.990329 7136 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0ab1515f9b343bd9d5c01bf6a8ee22c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0ab1515f9b343bd9d5c01bf6a8ee22c. Latest consensus state: current_term: 1 leader_uuid: "b0ab1515f9b343bd9d5c01bf6a8ee22c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ab1515f9b343bd9d5c01bf6a8ee22c" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37813 } } }
I20260929 10:53:56.990473 7136 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0ab1515f9b343bd9d5c01bf6a8ee22c [sys.catalog]: This master's current role is: LEADER
I20260929 10:53:56.990665 7135 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0ab1515f9b343bd9d5c01bf6a8ee22c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b0ab1515f9b343bd9d5c01bf6a8ee22c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ab1515f9b343bd9d5c01bf6a8ee22c" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 37813 } } }
I20260929 10:53:56.990782 7135 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0ab1515f9b343bd9d5c01bf6a8ee22c [sys.catalog]: This master's current role is: LEADER
W20260929 10:53:56.991623 7150 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b0ab1515f9b343bd9d5c01bf6a8ee22c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:53:56.991677 7150 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:53:56.991724 7143 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:53:56.991868 7143 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:53:56.993530 7143 catalog_manager.cc:1347] Generated new cluster ID: 09765061b0a149aaa0fe93396d77a7bc
I20260929 10:53:56.993583 7143 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:53:56.993825 7133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:56.987793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45344 (local address 127.20.80.254:37813)
0929 10:53:56.988005 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:56.988012 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:56.988036 (+ 24us) server_negotiation.cc:407] Connection header received
0929 10:53:56.988128 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:56.988133 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:56.988185 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:56.988324 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:53:56.988545 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.990201 (+ 1656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.991351 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:56.991878 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:56.992013 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:56.993037 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:53:56.993042 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:53:56.993047 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:53:56.993256 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:53:56.993298 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:53:56.993312 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:53:56.993384 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:56.993474 (+ 90us) server_negotiation.cc:299] Negotiation successful
0929 10:53:56.993649 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":96,"threads_started":1}
I20260929 10:53:57.020658 7143 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:53:57.021232 7143 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:53:57.024379 7143 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b0ab1515f9b343bd9d5c01bf6a8ee22c: Generated new TSK 0
I20260929 10:53:57.024569 7143 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:53:57.026491 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.193:0
--local_ip_for_outbound_sockets=127.20.80.193
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37813
--builtin_ntp_servers=127.20.80.212:33091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 10:53:57.107307 7154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:53:57.107542 7154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:53:57.107579 7154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:53:57.108727 7154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:53:57.108791 7154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:53:57.108837 7154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.193
I20260929 10:53:57.110237 7154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:33091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.193
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:53:57.110509 7154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:53:57.110817 7154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:53:57.115929 7159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:57.115934 7162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:57.115935 7160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:53:57.116257 7154 server_base.cc:1034] running on GCE node
I20260929 10:53:57.116489 7154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:53:57.116758 7154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:53:57.117942 7154 hybrid_clock.cc:648] HybridClock initialized: now 1790679237117920 us; error 32 us; skew 500 ppm
I20260929 10:53:57.119310 7154 webserver.cc:460] Webserver started at http://127.20.80.193:40103/ using document root <none> and password file <none>
I20260929 10:53:57.119606 7154 fs_manager.cc:359] Metadata directory not provided
I20260929 10:53:57.119668 7154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:53:57.119827 7154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:53:57.121342 7154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-0/data/instance:
uuid: "39177362846c4a86b540ffbf770ccc2e"
format_stamp: "Formatted at 2026-09-29 10:53:57 on dist-test-slave-vqch"
server_key: "5ffd225974e79a04fa10a5ea8a2e1092"
server_key_iv: "9fa445563dcebbde05adbd0d0e52fb14"
server_key_version: "encryptionkey@0"
I20260929 10:53:57.121760 7154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance:
uuid: "39177362846c4a86b540ffbf770ccc2e"
format_stamp: "Formatted at 2026-09-29 10:53:57 on dist-test-slave-vqch"
server_key: "5ffd225974e79a04fa10a5ea8a2e1092"
server_key_iv: "9fa445563dcebbde05adbd0d0e52fb14"
server_key_version: "encryptionkey@0"
I20260929 10:53:57.123273 7154 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:53:57.124884 7168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:57.125110 7154 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:53:57.125226 7154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-0/wal
uuid: "39177362846c4a86b540ffbf770ccc2e"
format_stamp: "Formatted at 2026-09-29 10:53:57 on dist-test-slave-vqch"
server_key: "5ffd225974e79a04fa10a5ea8a2e1092"
server_key_iv: "9fa445563dcebbde05adbd0d0e52fb14"
server_key_version: "encryptionkey@0"
I20260929 10:53:57.125303 7154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:53:57.150686 7154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:53:57.151024 7154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:53:57.151183 7154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:53:57.151427 7154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:53:57.151855 7154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:53:57.151914 7154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:57.151960 7154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:53:57.151988 7154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:57.157660 7154 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.193:40415
I20260929 10:53:57.157753 7281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.193:40415 every 8 connection(s)
I20260929 10:53:57.158087 7154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
I20260929 10:53:57.161801 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 7154
I20260929 10:53:57.161921 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance
I20260929 10:53:57.162123 20803 external_mini_cluster.cc:1428] Setting key 75d708735ecdb02ed03a8fc0a0043ab8
I20260929 10:53:57.163312 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.194:0
--local_ip_for_outbound_sockets=127.20.80.194
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37813
--builtin_ntp_servers=127.20.80.212:33091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 10:53:57.163530 7133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:57.158864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:50281 (local address 127.20.80.254:37813)
0929 10:53:57.158942 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:57.158944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:57.159140 (+ 196us) server_negotiation.cc:407] Connection header received
0929 10:53:57.159609 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:57.159610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:57.159623 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:57.159698 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:53:57.160367 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.160901 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.161939 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.162193 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.162282 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:57.163234 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:53:57.163235 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:53:57.163236 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:53:57.163241 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:53:57.163259 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:53:57.163264 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:53:57.163334 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:57.163419 (+ 85us) server_negotiation.cc:299] Negotiation successful
0929 10:53:57.163452 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 10:53:57.163776 7284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:57.158819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37813 (local address 127.20.80.193:50281)
0929 10:53:57.159003 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:57.159045 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:57.159527 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:57.159772 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:57.159776 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:57.159784 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:53:57.160287 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:57.160292 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.161037 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.161038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:57.161833 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:57.161835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.162293 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.162295 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:57.162462 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:57.163095 (+ 633us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:53:57.163096 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:53:57.163124 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:53:57.163148 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:53:57.163333 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:53:57.163335 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:53:57.163338 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:53:57.163419 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 10:53:57.163518 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":88,"threads_started":1}
I20260929 10:53:57.164280 7282 heartbeater.cc:347] Connected to a master server at 127.20.80.254:37813
I20260929 10:53:57.164358 7282 heartbeater.cc:464] Registering TS with master...
I20260929 10:53:57.164548 7282 heartbeater.cc:511] Master 127.20.80.254:37813 requested a full tablet report, sending...
I20260929 10:53:57.165094 7093 ts_manager.cc:194] Registered new tserver with Master: 39177362846c4a86b540ffbf770ccc2e (127.20.80.193:40415)
I20260929 10:53:57.165716 7093 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.80.193:50281
W20260929 10:53:57.243921 7285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:53:57.244110 7285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:53:57.244131 7285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:53:57.245308 7285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:53:57.245347 7285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:53:57.245373 7285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.194
I20260929 10:53:57.246712 7285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:33091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.194
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:53:57.246954 7285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:53:57.247280 7285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:53:57.252282 7290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:57.252287 7293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:57.252298 7291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:53:57.252494 7285 server_base.cc:1034] running on GCE node
I20260929 10:53:57.252794 7285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:53:57.253108 7285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:53:57.254276 7285 hybrid_clock.cc:648] HybridClock initialized: now 1790679237254242 us; error 52 us; skew 500 ppm
I20260929 10:53:57.255524 7285 webserver.cc:460] Webserver started at http://127.20.80.194:33451/ using document root <none> and password file <none>
I20260929 10:53:57.255788 7285 fs_manager.cc:359] Metadata directory not provided
I20260929 10:53:57.255863 7285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:53:57.256012 7285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:53:57.257133 7285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-1/data/instance:
uuid: "710839bd0b2c430a87277f90087f9c69"
format_stamp: "Formatted at 2026-09-29 10:53:57 on dist-test-slave-vqch"
server_key: "b781573e157804fd46d877dfaf3e1191"
server_key_iv: "10905f0156524eb6d2003299c0dcbc20"
server_key_version: "encryptionkey@0"
I20260929 10:53:57.257485 7285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance:
uuid: "710839bd0b2c430a87277f90087f9c69"
format_stamp: "Formatted at 2026-09-29 10:53:57 on dist-test-slave-vqch"
server_key: "b781573e157804fd46d877dfaf3e1191"
server_key_iv: "10905f0156524eb6d2003299c0dcbc20"
server_key_version: "encryptionkey@0"
I20260929 10:53:57.258762 7285 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:53:57.259497 7300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:57.259720 7285 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:53:57.259825 7285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-1/wal
uuid: "710839bd0b2c430a87277f90087f9c69"
format_stamp: "Formatted at 2026-09-29 10:53:57 on dist-test-slave-vqch"
server_key: "b781573e157804fd46d877dfaf3e1191"
server_key_iv: "10905f0156524eb6d2003299c0dcbc20"
server_key_version: "encryptionkey@0"
I20260929 10:53:57.259900 7285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:53:57.275168 7285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:53:57.275485 7285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:53:57.275645 7285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:53:57.275877 7285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:53:57.276203 7285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:53:57.276258 7285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:57.276298 7285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:53:57.276327 7285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:57.282088 7285 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.194:44503
I20260929 10:53:57.282135 7413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.194:44503 every 8 connection(s)
I20260929 10:53:57.282454 7285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
I20260929 10:53:57.287892 7133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:57.283300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:43145 (local address 127.20.80.254:37813)
0929 10:53:57.283390 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:57.283391 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:57.283641 (+ 250us) server_negotiation.cc:407] Connection header received
0929 10:53:57.284129 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:57.284131 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:57.284141 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:57.284199 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:53:57.285007 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.285513 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.286567 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.286755 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.286819 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:57.287630 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:53:57.287632 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:53:57.287632 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:53:57.287636 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:53:57.287651 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:53:57.287654 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:53:57.287707 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:57.287810 (+ 103us) server_negotiation.cc:299] Negotiation successful
0929 10:53:57.287833 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 10:53:57.288081 7416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:57.283263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37813 (local address 127.20.80.194:43145)
0929 10:53:57.283475 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:57.283518 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:57.284039 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:57.284267 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:57.284271 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:57.284280 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:53:57.284825 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:57.284831 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.285603 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.285603 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:57.286482 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:57.286485 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.286838 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.286839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:57.286951 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:57.287474 (+ 523us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:53:57.287475 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:53:57.287506 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:53:57.287531 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:53:57.287722 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:53:57.287726 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:53:57.287729 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:53:57.287786 (+ 57us) client_negotiation.cc:240] Negotiation successful
0929 10:53:57.287867 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":97,"threads_started":1}
I20260929 10:53:57.288640 7414 heartbeater.cc:347] Connected to a master server at 127.20.80.254:37813
I20260929 10:53:57.288750 7414 heartbeater.cc:464] Registering TS with master...
I20260929 10:53:57.288830 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 7285
I20260929 10:53:57.288935 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance
I20260929 10:53:57.289005 7414 heartbeater.cc:511] Master 127.20.80.254:37813 requested a full tablet report, sending...
I20260929 10:53:57.289187 20803 external_mini_cluster.cc:1428] Setting key 9dab7d143f522ed76cf25df585143bbb
I20260929 10:53:57.289427 7093 ts_manager.cc:194] Registered new tserver with Master: 710839bd0b2c430a87277f90087f9c69 (127.20.80.194:44503)
I20260929 10:53:57.289986 7093 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.80.194:43145
I20260929 10:53:57.290417 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.195:0
--local_ip_for_outbound_sockets=127.20.80.195
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37813
--builtin_ntp_servers=127.20.80.212:33091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 10:53:57.374208 7417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:53:57.374404 7417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:53:57.374424 7417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:53:57.375485 7417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:53:57.375519 7417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:53:57.375542 7417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.195
I20260929 10:53:57.376791 7417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:33091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:37813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.195
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:53:57.376961 7417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:53:57.377274 7417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:53:57.382115 7423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:57.382121 7422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:57.382186 7425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:53:57.382391 7417 server_base.cc:1034] running on GCE node
I20260929 10:53:57.382617 7417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:53:57.382866 7417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:53:57.384023 7417 hybrid_clock.cc:648] HybridClock initialized: now 1790679237384010 us; error 31 us; skew 500 ppm
I20260929 10:53:57.385383 7417 webserver.cc:460] Webserver started at http://127.20.80.195:44759/ using document root <none> and password file <none>
I20260929 10:53:57.385669 7417 fs_manager.cc:359] Metadata directory not provided
I20260929 10:53:57.385720 7417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:53:57.385892 7417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:53:57.386946 7417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-2/data/instance:
uuid: "34e02519c67b40b6b53e856a4a8a9616"
format_stamp: "Formatted at 2026-09-29 10:53:57 on dist-test-slave-vqch"
server_key: "f7e59b486e224093fc2bb5d837b8ff9c"
server_key_iv: "23aff66cfde9412edb70b7ef876c8e6f"
server_key_version: "encryptionkey@0"
I20260929 10:53:57.387307 7417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance:
uuid: "34e02519c67b40b6b53e856a4a8a9616"
format_stamp: "Formatted at 2026-09-29 10:53:57 on dist-test-slave-vqch"
server_key: "f7e59b486e224093fc2bb5d837b8ff9c"
server_key_iv: "23aff66cfde9412edb70b7ef876c8e6f"
server_key_version: "encryptionkey@0"
I20260929 10:53:57.388932 7417 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 10:53:57.389823 7431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:57.390022 7417 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:53:57.390122 7417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-2/wal
uuid: "34e02519c67b40b6b53e856a4a8a9616"
format_stamp: "Formatted at 2026-09-29 10:53:57 on dist-test-slave-vqch"
server_key: "f7e59b486e224093fc2bb5d837b8ff9c"
server_key_iv: "23aff66cfde9412edb70b7ef876c8e6f"
server_key_version: "encryptionkey@0"
I20260929 10:53:57.390205 7417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:53:57.401489 7417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:53:57.401767 7417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:53:57.401907 7417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:53:57.402123 7417 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:53:57.402431 7417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:53:57.402484 7417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:57.402525 7417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:53:57.402554 7417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:57.407968 7417 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.195:35129
I20260929 10:53:57.408041 7545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.195:35129 every 8 connection(s)
I20260929 10:53:57.408373 7417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
I20260929 10:53:57.413679 7133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:57.409163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:52389 (local address 127.20.80.254:37813)
0929 10:53:57.409207 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:57.409209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:57.409445 (+ 236us) server_negotiation.cc:407] Connection header received
0929 10:53:57.409990 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:57.409992 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:57.410006 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:57.410055 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:53:57.410724 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.411244 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.412221 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.412452 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.412542 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:57.413389 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:53:57.413390 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:53:57.413390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:53:57.413396 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:53:57.413410 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:53:57.413414 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:53:57.413468 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:57.413568 (+ 100us) server_negotiation.cc:299] Negotiation successful
0929 10:53:57.413599 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 10:53:57.413928 7548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:57.409154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:37813 (local address 127.20.80.195:52389)
0929 10:53:57.409316 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:57.409348 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:57.409870 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:57.410127 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:57.410131 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:57.410139 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:53:57.410620 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:57.410624 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.411348 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.411349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:57.412125 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:57.412127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.412553 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.412554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:57.412689 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:57.413240 (+ 551us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:53:57.413240 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:53:57.413268 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:53:57.413290 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:53:57.413471 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:53:57.413475 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:53:57.413479 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:53:57.413557 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 10:53:57.413715 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":72,"threads_started":1}
I20260929 10:53:57.414412 7546 heartbeater.cc:347] Connected to a master server at 127.20.80.254:37813
I20260929 10:53:57.414487 7546 heartbeater.cc:464] Registering TS with master...
I20260929 10:53:57.414693 7546 heartbeater.cc:511] Master 127.20.80.254:37813 requested a full tablet report, sending...
I20260929 10:53:57.415068 7093 ts_manager.cc:194] Registered new tserver with Master: 34e02519c67b40b6b53e856a4a8a9616 (127.20.80.195:35129)
I20260929 10:53:57.415454 7093 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.80.195:52389
I20260929 10:53:57.415576 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 7417
I20260929 10:53:57.415681 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance
I20260929 10:53:57.415913 20803 external_mini_cluster.cc:1428] Setting key ddcfb16244086ab9d6019ff21d92d5b6
I20260929 10:53:57.416765 20803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:53:57.420791 7133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:57.418173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45360 (local address 127.20.80.254:37813)
0929 10:53:57.418253 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:57.418256 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:57.418267 (+ 11us) server_negotiation.cc:407] Connection header received
0929 10:53:57.418332 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:57.418333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:57.418343 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:57.418382 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:53:57.418539 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.418975 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.419704 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.419885 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.419935 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:57.420526 (+ 591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:53:57.420527 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:53:57.420528 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:53:57.420531 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:53:57.420542 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:53:57.420546 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:53:57.420613 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:57.420674 (+ 61us) server_negotiation.cc:299] Negotiation successful
0929 10:53:57.420683 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 10:53:57.422223 7093 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45360:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:53:57.423785 7093 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:53:57.430437 7559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:57.426461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41794 (local address 127.20.80.195:35129)
0929 10:53:57.426610 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:57.426614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:57.426744 (+ 130us) server_negotiation.cc:407] Connection header received
0929 10:53:57.426781 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:57.426783 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:57.426797 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:57.426867 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:53:57.427031 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.428136 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.429263 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.429970 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.430053 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:57.430147 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:57.430327 (+ 180us) server_negotiation.cc:299] Negotiation successful
0929 10:53:57.430364 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":89,"threads_started":1}
I20260929 10:53:57.430471 7556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:57.426191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:40415 (local address 127.0.0.1:56914)
0929 10:53:57.426319 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:57.426335 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:57.426393 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:57.426691 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:57.426694 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:57.426697 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:53:57.426764 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:57.426767 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.428156 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.428157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:57.429273 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:57.429276 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.430119 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.430120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:57.430198 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:57.430211 (+ 13us) client_negotiation.cc:769] Sending connection context
0929 10:53:57.430297 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 10:53:57.430370 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":53,"thread_start_us":83,"threads_started":1}
I20260929 10:53:57.430471 7558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:57.426369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:35129 (local address 127.0.0.1:41794)
0929 10:53:57.426629 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:57.426639 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:57.426691 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:57.426924 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:57.426925 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:57.426927 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:53:57.426961 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:57.426962 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.428193 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.428194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:57.429180 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:57.429183 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.430106 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.430107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:57.430198 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:57.430211 (+ 13us) client_negotiation.cc:769] Sending connection context
0929 10:53:57.430297 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 10:53:57.430370 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":7,"thread_start_us":52,"threads_started":1}
I20260929 10:53:57.430684 7557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:57.426299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56914 (local address 127.20.80.193:40415)
0929 10:53:57.426507 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:57.426512 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:57.426524 (+ 12us) server_negotiation.cc:407] Connection header received
0929 10:53:57.426566 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:57.426568 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:57.426584 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:57.426666 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:53:57.426884 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.428054 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.429349 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.430040 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.430105 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:57.430179 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:57.430559 (+ 380us) server_negotiation.cc:299] Negotiation successful
0929 10:53:57.430610 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":90,"threads_started":1}
I20260929 10:53:57.431372 7480 tablet_service.cc:1467] Processing CreateTablet for tablet fef60c0743f040cf863bf7ee00d3e186 (DEFAULT_TABLE table=test-table [id=d590d723cf844748ba2d05731852d6e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:53:57.431372 7216 tablet_service.cc:1467] Processing CreateTablet for tablet fef60c0743f040cf863bf7ee00d3e186 (DEFAULT_TABLE table=test-table [id=d590d723cf844748ba2d05731852d6e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:53:57.431728 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:57.425965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:44503 (local address 127.0.0.1:39714)
0929 10:53:57.426104 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:57.426144 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:57.426213 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:57.426715 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:57.426716 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:57.426719 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:53:57.426774 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:57.426777 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.427882 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.427883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:57.429278 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:57.429280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.431510 (+ 2230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.431511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:57.431571 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:57.431574 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:53:57.431618 (+ 44us) client_negotiation.cc:240] Negotiation successful
0929 10:53:57.431660 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":66,"threads_started":1}
I20260929 10:53:57.431740 7555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:57.426092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39714 (local address 127.20.80.194:44503)
0929 10:53:57.426512 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:57.426516 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:57.426525 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:53:57.426566 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:57.426568 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:57.426584 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:57.426875 (+ 291us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:53:57.426943 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.427790 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.430222 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.431417 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.431508 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:57.431596 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:57.431628 (+ 32us) server_negotiation.cc:299] Negotiation successful
0929 10:53:57.431660 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":91,"threads_started":1}
I20260929 10:53:57.431743 7480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fef60c0743f040cf863bf7ee00d3e186. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:53:57.431743 7216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fef60c0743f040cf863bf7ee00d3e186. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:53:57.432314 7348 tablet_service.cc:1467] Processing CreateTablet for tablet fef60c0743f040cf863bf7ee00d3e186 (DEFAULT_TABLE table=test-table [id=d590d723cf844748ba2d05731852d6e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:53:57.432564 7348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fef60c0743f040cf863bf7ee00d3e186. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:53:57.435077 7561 tablet_bootstrap.cc:492] T fef60c0743f040cf863bf7ee00d3e186 P 34e02519c67b40b6b53e856a4a8a9616: Bootstrap starting.
I20260929 10:53:57.435328 7562 tablet_bootstrap.cc:492] T fef60c0743f040cf863bf7ee00d3e186 P 710839bd0b2c430a87277f90087f9c69: Bootstrap starting.
I20260929 10:53:57.435711 7561 tablet_bootstrap.cc:654] T fef60c0743f040cf863bf7ee00d3e186 P 34e02519c67b40b6b53e856a4a8a9616: Neither blocks nor log segments found. Creating new log.
I20260929 10:53:57.435917 7562 tablet_bootstrap.cc:654] T fef60c0743f040cf863bf7ee00d3e186 P 710839bd0b2c430a87277f90087f9c69: Neither blocks nor log segments found. Creating new log.
I20260929 10:53:57.436046 7561 log.cc:824] T fef60c0743f040cf863bf7ee00d3e186 P 34e02519c67b40b6b53e856a4a8a9616: Log is configured to *not* fsync() on all Append() calls
I20260929 10:53:57.436269 7562 log.cc:824] T fef60c0743f040cf863bf7ee00d3e186 P 710839bd0b2c430a87277f90087f9c69: Log is configured to *not* fsync() on all Append() calls
I20260929 10:53:57.436380 7560 tablet_bootstrap.cc:492] T fef60c0743f040cf863bf7ee00d3e186 P 39177362846c4a86b540ffbf770ccc2e: Bootstrap starting.
I20260929 10:53:57.436771 7561 tablet_bootstrap.cc:492] T fef60c0743f040cf863bf7ee00d3e186 P 34e02519c67b40b6b53e856a4a8a9616: No bootstrap required, opened a new log
I20260929 10:53:57.436821 7561 ts_tablet_manager.cc:1397] T fef60c0743f040cf863bf7ee00d3e186 P 34e02519c67b40b6b53e856a4a8a9616: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:53:57.436961 7560 tablet_bootstrap.cc:654] T fef60c0743f040cf863bf7ee00d3e186 P 39177362846c4a86b540ffbf770ccc2e: Neither blocks nor log segments found. Creating new log.
I20260929 10:53:57.437139 7562 tablet_bootstrap.cc:492] T fef60c0743f040cf863bf7ee00d3e186 P 710839bd0b2c430a87277f90087f9c69: No bootstrap required, opened a new log
I20260929 10:53:57.437206 7562 ts_tablet_manager.cc:1397] T fef60c0743f040cf863bf7ee00d3e186 P 710839bd0b2c430a87277f90087f9c69: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:53:57.437330 7560 log.cc:824] T fef60c0743f040cf863bf7ee00d3e186 P 39177362846c4a86b540ffbf770ccc2e: Log is configured to *not* fsync() on all Append() calls
I20260929 10:53:57.438227 7560 tablet_bootstrap.cc:492] T fef60c0743f040cf863bf7ee00d3e186 P 39177362846c4a86b540ffbf770ccc2e: No bootstrap required, opened a new log
I20260929 10:53:57.438280 7560 ts_tablet_manager.cc:1397] T fef60c0743f040cf863bf7ee00d3e186 P 39177362846c4a86b540ffbf770ccc2e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260929 10:53:57.438851 7561 raft_consensus.cc:348] T fef60c0743f040cf863bf7ee00d3e186 P 34e02519c67b40b6b53e856a4a8a9616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710839bd0b2c430a87277f90087f9c69" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 44503 } } peers { permanent_uuid: "39177362846c4a86b540ffbf770ccc2e" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 40415 } } peers { permanent_uuid: "34e02519c67b40b6b53e856a4a8a9616" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35129 } }
I20260929 10:53:57.438966 7561 raft_consensus.cc:374] T fef60c0743f040cf863bf7ee00d3e186 P 34e02519c67b40b6b53e856a4a8a9616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:53:57.438987 7561 raft_consensus.cc:729] T fef60c0743f040cf863bf7ee00d3e186 P 34e02519c67b40b6b53e856a4a8a9616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34e02519c67b40b6b53e856a4a8a9616, State: Initialized, Role: FOLLOWER
I20260929 10:53:57.439107 7561 consensus_queue.cc:260] T fef60c0743f040cf863bf7ee00d3e186 P 34e02519c67b40b6b53e856a4a8a9616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710839bd0b2c430a87277f90087f9c69" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 44503 } } peers { permanent_uuid: "39177362846c4a86b540ffbf770ccc2e" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 40415 } } peers { permanent_uuid: "34e02519c67b40b6b53e856a4a8a9616" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35129 } }
I20260929 10:53:57.439383 7546 heartbeater.cc:503] Master 127.20.80.254:37813 was elected leader, sending a full tablet report...
I20260929 10:53:57.439586 7561 ts_tablet_manager.cc:1428] T fef60c0743f040cf863bf7ee00d3e186 P 34e02519c67b40b6b53e856a4a8a9616: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 10:53:57.439821 7562 raft_consensus.cc:348] T fef60c0743f040cf863bf7ee00d3e186 P 710839bd0b2c430a87277f90087f9c69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710839bd0b2c430a87277f90087f9c69" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 44503 } } peers { permanent_uuid: "39177362846c4a86b540ffbf770ccc2e" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 40415 } } peers { permanent_uuid: "34e02519c67b40b6b53e856a4a8a9616" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35129 } }
I20260929 10:53:57.439965 7562 raft_consensus.cc:374] T fef60c0743f040cf863bf7ee00d3e186 P 710839bd0b2c430a87277f90087f9c69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:53:57.440003 7562 raft_consensus.cc:729] T fef60c0743f040cf863bf7ee00d3e186 P 710839bd0b2c430a87277f90087f9c69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 710839bd0b2c430a87277f90087f9c69, State: Initialized, Role: FOLLOWER
I20260929 10:53:57.440119 7562 consensus_queue.cc:260] T fef60c0743f040cf863bf7ee00d3e186 P 710839bd0b2c430a87277f90087f9c69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710839bd0b2c430a87277f90087f9c69" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 44503 } } peers { permanent_uuid: "39177362846c4a86b540ffbf770ccc2e" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 40415 } } peers { permanent_uuid: "34e02519c67b40b6b53e856a4a8a9616" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35129 } }
I20260929 10:53:57.440409 7414 heartbeater.cc:503] Master 127.20.80.254:37813 was elected leader, sending a full tablet report...
I20260929 10:53:57.440371 7562 ts_tablet_manager.cc:1428] T fef60c0743f040cf863bf7ee00d3e186 P 710839bd0b2c430a87277f90087f9c69: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 10:53:57.440807 7560 raft_consensus.cc:348] T fef60c0743f040cf863bf7ee00d3e186 P 39177362846c4a86b540ffbf770ccc2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710839bd0b2c430a87277f90087f9c69" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 44503 } } peers { permanent_uuid: "39177362846c4a86b540ffbf770ccc2e" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 40415 } } peers { permanent_uuid: "34e02519c67b40b6b53e856a4a8a9616" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35129 } }
I20260929 10:53:57.440963 7560 raft_consensus.cc:374] T fef60c0743f040cf863bf7ee00d3e186 P 39177362846c4a86b540ffbf770ccc2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:53:57.441037 7560 raft_consensus.cc:729] T fef60c0743f040cf863bf7ee00d3e186 P 39177362846c4a86b540ffbf770ccc2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39177362846c4a86b540ffbf770ccc2e, State: Initialized, Role: FOLLOWER
I20260929 10:53:57.441144 7560 consensus_queue.cc:260] T fef60c0743f040cf863bf7ee00d3e186 P 39177362846c4a86b540ffbf770ccc2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710839bd0b2c430a87277f90087f9c69" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 44503 } } peers { permanent_uuid: "39177362846c4a86b540ffbf770ccc2e" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 40415 } } peers { permanent_uuid: "34e02519c67b40b6b53e856a4a8a9616" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35129 } }
I20260929 10:53:57.441407 7560 ts_tablet_manager.cc:1428] T fef60c0743f040cf863bf7ee00d3e186 P 39177362846c4a86b540ffbf770ccc2e: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260929 10:53:57.441485 7282 heartbeater.cc:503] Master 127.20.80.254:37813 was elected leader, sending a full tablet report...
W20260929 10:53:57.533061 7415 tablet.cc:2369] T fef60c0743f040cf863bf7ee00d3e186 P 710839bd0b2c430a87277f90087f9c69: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:53:57.658864 7283 tablet.cc:2369] T fef60c0743f040cf863bf7ee00d3e186 P 39177362846c4a86b540ffbf770ccc2e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:53:57.658954 7547 tablet.cc:2369] T fef60c0743f040cf863bf7ee00d3e186 P 34e02519c67b40b6b53e856a4a8a9616: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:53:57.751240 7568 raft_consensus.cc:482] T fef60c0743f040cf863bf7ee00d3e186 P 39177362846c4a86b540ffbf770ccc2e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:53:57.751376 7568 raft_consensus.cc:504] T fef60c0743f040cf863bf7ee00d3e186 P 39177362846c4a86b540ffbf770ccc2e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710839bd0b2c430a87277f90087f9c69" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 44503 } } peers { permanent_uuid: "39177362846c4a86b540ffbf770ccc2e" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 40415 } } peers { permanent_uuid: "34e02519c67b40b6b53e856a4a8a9616" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35129 } }
I20260929 10:53:57.751866 7568 leader_election.cc:288] T fef60c0743f040cf863bf7ee00d3e186 P 39177362846c4a86b540ffbf770ccc2e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 710839bd0b2c430a87277f90087f9c69 (127.20.80.194:44503), 34e02519c67b40b6b53e856a4a8a9616 (127.20.80.195:35129)
I20260929 10:53:57.756035 7555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:57.752230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:46871 (local address 127.20.80.194:44503)
0929 10:53:57.752293 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:57.752295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:57.752310 (+ 15us) server_negotiation.cc:407] Connection header received
0929 10:53:57.752446 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:57.752448 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:57.752458 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:57.752492 (+ 34us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:53:57.752726 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.753518 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.755075 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.755638 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.755723 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:57.755729 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:57.755918 (+ 189us) server_negotiation.cc:299] Negotiation successful
0929 10:53:57.755962 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 10:53:57.756035 7559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:57.752384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:56643 (local address 127.20.80.195:35129)
0929 10:53:57.752463 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:57.752465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:57.752497 (+ 32us) server_negotiation.cc:407] Connection header received
0929 10:53:57.752575 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:57.752576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:57.752587 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:57.752643 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:53:57.752917 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.753647 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.755075 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.755638 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.755720 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:57.755729 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:57.755918 (+ 189us) server_negotiation.cc:299] Negotiation successful
0929 10:53:57.755962 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 10:53:57.756080 7570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:57.752292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:35129 (local address 127.20.80.193:56643)
0929 10:53:57.752426 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:57.752436 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:57.752513 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:57.752728 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:57.752729 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:57.752732 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:53:57.752825 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:57.752827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.753710 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.753710 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:57.754974 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:57.754977 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.755741 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.755742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:57.755824 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:57.755830 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 10:53:57.755898 (+ 68us) client_negotiation.cc:240] Negotiation successful
0929 10:53:57.755955 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":5,"thread_start_us":69,"threads_started":1}
I20260929 10:53:57.756080 7569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:57.752091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:44503 (local address 127.20.80.193:46871)
0929 10:53:57.752239 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:57.752250 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:57.752313 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:57.752576 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:57.752577 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:57.752579 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:53:57.752652 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:57.752654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.753599 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.753600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:57.754974 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:57.754977 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.755741 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.755742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:57.755843 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:57.755846 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:53:57.755897 (+ 51us) client_negotiation.cc:240] Negotiation successful
0929 10:53:57.755955 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":46,"thread_start_us":80,"threads_started":1}
I20260929 10:53:57.756515 7500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fef60c0743f040cf863bf7ee00d3e186" candidate_uuid: "39177362846c4a86b540ffbf770ccc2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34e02519c67b40b6b53e856a4a8a9616" is_pre_election: true
I20260929 10:53:57.756690 7500 raft_consensus.cc:2454] T fef60c0743f040cf863bf7ee00d3e186 P 34e02519c67b40b6b53e856a4a8a9616 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39177362846c4a86b540ffbf770ccc2e in term 0.
I20260929 10:53:57.757038 7170 leader_election.cc:302] T fef60c0743f040cf863bf7ee00d3e186 P 39177362846c4a86b540ffbf770ccc2e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34e02519c67b40b6b53e856a4a8a9616, 39177362846c4a86b540ffbf770ccc2e; no voters:
I20260929 10:53:57.757189 7568 raft_consensus.cc:2781] T fef60c0743f040cf863bf7ee00d3e186 P 39177362846c4a86b540ffbf770ccc2e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:53:57.757164 7368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fef60c0743f040cf863bf7ee00d3e186" candidate_uuid: "39177362846c4a86b540ffbf770ccc2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710839bd0b2c430a87277f90087f9c69" is_pre_election: true
I20260929 10:53:57.757241 7568 raft_consensus.cc:482] T fef60c0743f040cf863bf7ee00d3e186 P 39177362846c4a86b540ffbf770ccc2e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:53:57.757264 7568 raft_consensus.cc:3037] T fef60c0743f040cf863bf7ee00d3e186 P 39177362846c4a86b540ffbf770ccc2e [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:53:57.757314 7368 raft_consensus.cc:2454] T fef60c0743f040cf863bf7ee00d3e186 P 710839bd0b2c430a87277f90087f9c69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39177362846c4a86b540ffbf770ccc2e in term 0.
I20260929 10:53:57.758024 7568 raft_consensus.cc:504] T fef60c0743f040cf863bf7ee00d3e186 P 39177362846c4a86b540ffbf770ccc2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710839bd0b2c430a87277f90087f9c69" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 44503 } } peers { permanent_uuid: "39177362846c4a86b540ffbf770ccc2e" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 40415 } } peers { permanent_uuid: "34e02519c67b40b6b53e856a4a8a9616" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35129 } }
I20260929 10:53:57.758133 7568 leader_election.cc:288] T fef60c0743f040cf863bf7ee00d3e186 P 39177362846c4a86b540ffbf770ccc2e [CANDIDATE]: Term 1 election: Requested vote from peers 710839bd0b2c430a87277f90087f9c69 (127.20.80.194:44503), 34e02519c67b40b6b53e856a4a8a9616 (127.20.80.195:35129)
I20260929 10:53:57.758298 7368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fef60c0743f040cf863bf7ee00d3e186" candidate_uuid: "39177362846c4a86b540ffbf770ccc2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710839bd0b2c430a87277f90087f9c69"
I20260929 10:53:57.758378 7368 raft_consensus.cc:3037] T fef60c0743f040cf863bf7ee00d3e186 P 710839bd0b2c430a87277f90087f9c69 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:53:57.758420 7500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fef60c0743f040cf863bf7ee00d3e186" candidate_uuid: "39177362846c4a86b540ffbf770ccc2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34e02519c67b40b6b53e856a4a8a9616"
I20260929 10:53:57.758519 7500 raft_consensus.cc:3037] T fef60c0743f040cf863bf7ee00d3e186 P 34e02519c67b40b6b53e856a4a8a9616 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:53:57.759125 7368 raft_consensus.cc:2454] T fef60c0743f040cf863bf7ee00d3e186 P 710839bd0b2c430a87277f90087f9c69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39177362846c4a86b540ffbf770ccc2e in term 1.
I20260929 10:53:57.759287 7170 leader_election.cc:302] T fef60c0743f040cf863bf7ee00d3e186 P 39177362846c4a86b540ffbf770ccc2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39177362846c4a86b540ffbf770ccc2e, 710839bd0b2c430a87277f90087f9c69; no voters:
I20260929 10:53:57.759370 7568 raft_consensus.cc:2781] T fef60c0743f040cf863bf7ee00d3e186 P 39177362846c4a86b540ffbf770ccc2e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:53:57.759388 7500 raft_consensus.cc:2454] T fef60c0743f040cf863bf7ee00d3e186 P 34e02519c67b40b6b53e856a4a8a9616 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39177362846c4a86b540ffbf770ccc2e in term 1.
I20260929 10:53:57.759533 7568 raft_consensus.cc:686] T fef60c0743f040cf863bf7ee00d3e186 P 39177362846c4a86b540ffbf770ccc2e [term 1 LEADER]: Becoming Leader. State: Replica: 39177362846c4a86b540ffbf770ccc2e, State: Running, Role: LEADER
I20260929 10:53:57.759630 7568 consensus_queue.cc:237] T fef60c0743f040cf863bf7ee00d3e186 P 39177362846c4a86b540ffbf770ccc2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710839bd0b2c430a87277f90087f9c69" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 44503 } } peers { permanent_uuid: "39177362846c4a86b540ffbf770ccc2e" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 40415 } } peers { permanent_uuid: "34e02519c67b40b6b53e856a4a8a9616" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35129 } }
I20260929 10:53:57.760512 7093 catalog_manager.cc:5495] T fef60c0743f040cf863bf7ee00d3e186 P 39177362846c4a86b540ffbf770ccc2e reported cstate change: term changed from 0 to 1, leader changed from <none> to 39177362846c4a86b540ffbf770ccc2e (127.20.80.193). New cstate: current_term: 1 leader_uuid: "39177362846c4a86b540ffbf770ccc2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710839bd0b2c430a87277f90087f9c69" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 44503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39177362846c4a86b540ffbf770ccc2e" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34e02519c67b40b6b53e856a4a8a9616" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35129 } health_report { overall_health: UNKNOWN } } }
I20260929 10:53:57.782806 7557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:57.780629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56918 (local address 127.20.80.193:40415)
0929 10:53:57.780702 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:57.780704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:57.780736 (+ 32us) server_negotiation.cc:407] Connection header received
0929 10:53:57.780783 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:57.780784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:57.780804 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:57.780860 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:53:57.781003 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.781474 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.782170 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:57.782356 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:57.782406 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:57.782529 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:53:57.782622 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:53:57.782692 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:57.782735 (+ 43us) server_negotiation.cc:299] Negotiation successful
0929 10:53:57.782744 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 10:53:57.784727 7368 raft_consensus.cc:1261] T fef60c0743f040cf863bf7ee00d3e186 P 710839bd0b2c430a87277f90087f9c69 [term 1 FOLLOWER]: Refusing update from remote peer 39177362846c4a86b540ffbf770ccc2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:53:57.784726 7500 raft_consensus.cc:1261] T fef60c0743f040cf863bf7ee00d3e186 P 34e02519c67b40b6b53e856a4a8a9616 [term 1 FOLLOWER]: Refusing update from remote peer 39177362846c4a86b540ffbf770ccc2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:53:57.785118 7568 consensus_queue.cc:1035] T fef60c0743f040cf863bf7ee00d3e186 P 39177362846c4a86b540ffbf770ccc2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "710839bd0b2c430a87277f90087f9c69" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 44503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:53:57.785264 7571 consensus_queue.cc:1035] T fef60c0743f040cf863bf7ee00d3e186 P 39177362846c4a86b540ffbf770ccc2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "34e02519c67b40b6b53e856a4a8a9616" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 35129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:53:57.787665 7574 mvcc.cc:205] Tried to move back new op lower bound from 7334622157963272192 to 7334622157864075264. Current Snapshot: MvccSnapshot[applied={T|T < 7334622157963272192}]
I20260929 10:53:57.788268 7576 mvcc.cc:205] Tried to move back new op lower bound from 7334622157963272192 to 7334622157864075264. Current Snapshot: MvccSnapshot[applied={T|T < 7334622157963272192}]
I20260929 10:53:57.790412 7093 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:53:57.790537 7093 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:53:57.791280 7093 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b0ab1515f9b343bd9d5c01bf6a8ee22c: Sending DeleteTablet for 3 replicas of tablet fef60c0743f040cf863bf7ee00d3e186
I20260929 10:53:57.791617 7348 tablet_service.cc:1514] Processing DeleteTablet for tablet fef60c0743f040cf863bf7ee00d3e186 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:53:57 UTC) from {username='slave'} at 127.0.0.1:39714
I20260929 10:53:57.791797 7216 tablet_service.cc:1514] Processing DeleteTablet for tablet fef60c0743f040cf863bf7ee00d3e186 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:53:57 UTC) from {username='slave'} at 127.0.0.1:56914
I20260929 10:53:57.791810 7480 tablet_service.cc:1514] Processing DeleteTablet for tablet fef60c0743f040cf863bf7ee00d3e186 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:53:57 UTC) from {username='slave'} at 127.0.0.1:41794
I20260929 10:53:57.791828 7584 tablet_replica.cc:330] T fef60c0743f040cf863bf7ee00d3e186 P 710839bd0b2c430a87277f90087f9c69: stopping tablet replica
I20260929 10:53:57.791922 7584 raft_consensus.cc:2229] T fef60c0743f040cf863bf7ee00d3e186 P 710839bd0b2c430a87277f90087f9c69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:53:57.792014 7584 raft_consensus.cc:2258] T fef60c0743f040cf863bf7ee00d3e186 P 710839bd0b2c430a87277f90087f9c69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:53:57.792037 7585 tablet_replica.cc:330] T fef60c0743f040cf863bf7ee00d3e186 P 34e02519c67b40b6b53e856a4a8a9616: stopping tablet replica
I20260929 10:53:57.792120 7585 raft_consensus.cc:2229] T fef60c0743f040cf863bf7ee00d3e186 P 34e02519c67b40b6b53e856a4a8a9616 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:53:57.792232 7585 raft_consensus.cc:2258] T fef60c0743f040cf863bf7ee00d3e186 P 34e02519c67b40b6b53e856a4a8a9616 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:53:57.792415 7586 tablet_replica.cc:330] T fef60c0743f040cf863bf7ee00d3e186 P 39177362846c4a86b540ffbf770ccc2e: stopping tablet replica
I20260929 10:53:57.792500 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 7154
I20260929 10:53:57.792553 7584 ts_tablet_manager.cc:1905] T fef60c0743f040cf863bf7ee00d3e186 P 710839bd0b2c430a87277f90087f9c69: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:53:57.793704 7585 ts_tablet_manager.cc:1905] T fef60c0743f040cf863bf7ee00d3e186 P 34e02519c67b40b6b53e856a4a8a9616: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:53:57.794726 7584 ts_tablet_manager.cc:1918] T fef60c0743f040cf863bf7ee00d3e186 P 710839bd0b2c430a87277f90087f9c69: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:53:57.794803 7584 log.cc:1196] T fef60c0743f040cf863bf7ee00d3e186 P 710839bd0b2c430a87277f90087f9c69: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-1/wal/wals/fef60c0743f040cf863bf7ee00d3e186
I20260929 10:53:57.795101 7584 ts_tablet_manager.cc:1939] T fef60c0743f040cf863bf7ee00d3e186 P 710839bd0b2c430a87277f90087f9c69: Deleting consensus metadata
I20260929 10:53:57.795398 7585 ts_tablet_manager.cc:1918] T fef60c0743f040cf863bf7ee00d3e186 P 34e02519c67b40b6b53e856a4a8a9616: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:53:57.795486 7585 log.cc:1196] T fef60c0743f040cf863bf7ee00d3e186 P 34e02519c67b40b6b53e856a4a8a9616: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790678996126930-20803-0/minicluster-data/ts-2/wal/wals/fef60c0743f040cf863bf7ee00d3e186
I20260929 10:53:57.795668 7078 catalog_manager.cc:4841] TS 710839bd0b2c430a87277f90087f9c69 (127.20.80.194:44503): tablet fef60c0743f040cf863bf7ee00d3e186 (table test-table [id=d590d723cf844748ba2d05731852d6e7]) successfully deleted
I20260929 10:53:57.795771 7585 ts_tablet_manager.cc:1939] T fef60c0743f040cf863bf7ee00d3e186 P 34e02519c67b40b6b53e856a4a8a9616: Deleting consensus metadata
I20260929 10:53:57.796224 7078 catalog_manager.cc:4841] TS 34e02519c67b40b6b53e856a4a8a9616 (127.20.80.195:35129): tablet fef60c0743f040cf863bf7ee00d3e186 (table test-table [id=d590d723cf844748ba2d05731852d6e7]) successfully deleted
W20260929 10:53:57.798637 7078 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.80.193:40415 (error 108)
I20260929 10:53:57.799014 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 7285
I20260929 10:53:57.799162 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:57.799010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:40415 (local address 127.0.0.1:56924)
0929 10:53:57.799075 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:57.799124 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.193:40415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260929 10:53:57.800000 7078 catalog_manager.cc:4584] TS 39177362846c4a86b540ffbf770ccc2e (127.20.80.193:40415): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fef60c0743f040cf863bf7ee00d3e186: Network error: Client connection negotiation failed: client connection to 127.20.80.193:40415: connect: Connection refused (error 111)
I20260929 10:53:57.804358 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 7417
I20260929 10:53:57.808641 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 7062
2026-09-29T10:53:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (984 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:53:57 dist-test-slave-vqch krb5kdc[7590](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:53:57 dist-test-slave-vqch krb5kdc[7590](info): set up 2 sockets
Sep 29 10:53:57 dist-test-slave-vqch krb5kdc[7590](info): commencing operation
krb5kdc: starting...
W20260929 10:53:58.884474 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:53:58 dist-test-slave-vqch krb5kdc[7590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:53:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:53:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.80.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
I20260929 10:53:59.009382 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:44355
--webserver_interface=127.20.80.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:45981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:44355
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:53:59.090462 7603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:53:59.090667 7603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:53:59.090687 7603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:53:59.091827 7603 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:53:59.091869 7603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:53:59.091883 7603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:53:59.091897 7603 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:53:59.091910 7603 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:53:59.093413 7603 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:45981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:44355
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:44355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:53:59.093660 7603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:53:59.093906 7603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:53:59.098878 7608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:59.098894 7609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:59.098942 7611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:53:59.099211 7603 server_base.cc:1034] running on GCE node
I20260929 10:53:59.099475 7603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:53:59.099758 7603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:53:59.100925 7603 hybrid_clock.cc:648] HybridClock initialized: now 1790679239100910 us; error 46 us; skew 500 ppm
Sep 29 10:53:59 dist-test-slave-vqch krb5kdc[7590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679239, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:53:59.103639 7603 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:53:59.104202 7603 webserver.cc:460] Webserver started at http://127.20.80.254:42357/ using document root <none> and password file <none>
I20260929 10:53:59.104435 7603 fs_manager.cc:359] Metadata directory not provided
I20260929 10:53:59.104519 7603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:53:59.104691 7603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:53:59.105872 7603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "15732e87f56a4f6dafe7bbd5875f9719"
format_stamp: "Formatted at 2026-09-29 10:53:59 on dist-test-slave-vqch"
server_key: "3ddfdee2ef17485d1b90a3ff27e6a16b"
server_key_iv: "ab7242a11f6403a5145478334528a639"
server_key_version: "encryptionkey@0"
I20260929 10:53:59.106271 7603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "15732e87f56a4f6dafe7bbd5875f9719"
format_stamp: "Formatted at 2026-09-29 10:53:59 on dist-test-slave-vqch"
server_key: "3ddfdee2ef17485d1b90a3ff27e6a16b"
server_key_iv: "ab7242a11f6403a5145478334528a639"
server_key_version: "encryptionkey@0"
I20260929 10:53:59.107748 7603 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 10:53:59.108525 7618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:59.108817 7603 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:53:59.108933 7603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "15732e87f56a4f6dafe7bbd5875f9719"
format_stamp: "Formatted at 2026-09-29 10:53:59 on dist-test-slave-vqch"
server_key: "3ddfdee2ef17485d1b90a3ff27e6a16b"
server_key_iv: "ab7242a11f6403a5145478334528a639"
server_key_version: "encryptionkey@0"
I20260929 10:53:59.109038 7603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:53:59.132263 7603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:53:59.134766 7603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:53:59.134935 7603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:53:59.139055 7603 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:44355
I20260929 10:53:59.139094 7670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:44355 every 8 connection(s)
I20260929 10:53:59.139478 7603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:53:59.140352 7671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:53:59.142580 7671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15732e87f56a4f6dafe7bbd5875f9719: Bootstrap starting.
I20260929 10:53:59.143113 7671 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15732e87f56a4f6dafe7bbd5875f9719: Neither blocks nor log segments found. Creating new log.
I20260929 10:53:59.143350 7671 log.cc:824] T 00000000000000000000000000000000 P 15732e87f56a4f6dafe7bbd5875f9719: Log is configured to *not* fsync() on all Append() calls
I20260929 10:53:59.143978 7671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15732e87f56a4f6dafe7bbd5875f9719: No bootstrap required, opened a new log
I20260929 10:53:59.144560 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 7603
I20260929 10:53:59.144701 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:53:59.144893 20803 external_mini_cluster.cc:1428] Setting key 17f5f4c8c53d627731ba89d50dcc8b41
I20260929 10:53:59.146184 7671 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15732e87f56a4f6dafe7bbd5875f9719 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15732e87f56a4f6dafe7bbd5875f9719" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44355 } }
I20260929 10:53:59.146348 7671 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15732e87f56a4f6dafe7bbd5875f9719 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:53:59.146389 7671 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15732e87f56a4f6dafe7bbd5875f9719 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15732e87f56a4f6dafe7bbd5875f9719, State: Initialized, Role: FOLLOWER
I20260929 10:53:59.146557 7671 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15732e87f56a4f6dafe7bbd5875f9719 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15732e87f56a4f6dafe7bbd5875f9719" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44355 } }
I20260929 10:53:59.146648 7671 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15732e87f56a4f6dafe7bbd5875f9719 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:53:59.146715 7671 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15732e87f56a4f6dafe7bbd5875f9719 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:53:59.146775 7671 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15732e87f56a4f6dafe7bbd5875f9719 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:53:59.147442 7671 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15732e87f56a4f6dafe7bbd5875f9719 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15732e87f56a4f6dafe7bbd5875f9719" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44355 } }
I20260929 10:53:59.147532 7671 leader_election.cc:302] T 00000000000000000000000000000000 P 15732e87f56a4f6dafe7bbd5875f9719 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15732e87f56a4f6dafe7bbd5875f9719; no voters:
I20260929 10:53:59.147724 7671 leader_election.cc:288] T 00000000000000000000000000000000 P 15732e87f56a4f6dafe7bbd5875f9719 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:53:59.147746 7676 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15732e87f56a4f6dafe7bbd5875f9719 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:53:59.147857 7676 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15732e87f56a4f6dafe7bbd5875f9719 [term 1 LEADER]: Becoming Leader. State: Replica: 15732e87f56a4f6dafe7bbd5875f9719, State: Running, Role: LEADER
I20260929 10:53:59.148000 7671 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15732e87f56a4f6dafe7bbd5875f9719 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:53:59.148000 7676 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15732e87f56a4f6dafe7bbd5875f9719 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15732e87f56a4f6dafe7bbd5875f9719" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44355 } }
I20260929 10:53:59.149770 7677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15732e87f56a4f6dafe7bbd5875f9719 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15732e87f56a4f6dafe7bbd5875f9719" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15732e87f56a4f6dafe7bbd5875f9719" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44355 } } }
I20260929 10:53:59.149748 7678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15732e87f56a4f6dafe7bbd5875f9719 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15732e87f56a4f6dafe7bbd5875f9719. Latest consensus state: current_term: 1 leader_uuid: "15732e87f56a4f6dafe7bbd5875f9719" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15732e87f56a4f6dafe7bbd5875f9719" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44355 } } }
W20260929 10:53:59.149986 7692 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 15732e87f56a4f6dafe7bbd5875f9719: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260929 10:53:59.150007 7678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15732e87f56a4f6dafe7bbd5875f9719 [sys.catalog]: This master's current role is: LEADER
I20260929 10:53:59.149928 7677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15732e87f56a4f6dafe7bbd5875f9719 [sys.catalog]: This master's current role is: LEADER
W20260929 10:53:59.150063 7692 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:53:59.150273 7694 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:53:59.150453 7694 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 29 10:53:59 dist-test-slave-vqch krb5kdc[7590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:53:59.152525 7694 catalog_manager.cc:1347] Generated new cluster ID: 7a746aa25d704b0e97995f21c24d2db2
I20260929 10:53:59.152576 7694 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:53:59.156661 7675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:59.145451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34526 (local address 127.20.80.254:44355)
0929 10:53:59.145640 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:59.145648 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:59.145673 (+ 25us) server_negotiation.cc:407] Connection header received
0929 10:53:59.146167 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:59.146173 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:59.146220 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:59.146367 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:53:59.147170 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.148353 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.149421 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.150051 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.150148 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:59.152925 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:53:59.152928 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:53:59.152929 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:53:59.152940 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:53:59.155037 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:59.155400 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:59.155403 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:59.155404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:59.155537 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:59.155748 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:59.155749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:59.155750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:59.156064 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:53:59.156227 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:59.156309 (+ 82us) server_negotiation.cc:299] Negotiation successful
0929 10:53:59.156450 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":91,"threads_started":1}
I20260929 10:53:59.189872 7694 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:53:59.190456 7694 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:53:59.194507 7694 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15732e87f56a4f6dafe7bbd5875f9719: Generated new TSK 0
I20260929 10:53:59.194702 7694 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.80.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
Entry for principal kudu/127.20.80.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab.
I20260929 10:53:59.223752 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.193:0
--local_ip_for_outbound_sockets=127.20.80.193
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:44355
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:45981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:53:59.305686 7698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:53:59.305928 7698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:53:59.305972 7698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:53:59.307224 7698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:53:59.307296 7698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:53:59.307341 7698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.193
I20260929 10:53:59.308867 7698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:45981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.193.keytab
--principal=kudu/127.20.80.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:44355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.193
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:53:59.309154 7698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:53:59.309383 7698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:53:59.314322 7706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:59.314399 7703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:59.314319 7704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:53:59.314471 7698 server_base.cc:1034] running on GCE node
I20260929 10:53:59.314779 7698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:53:59.315095 7698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:53:59.316252 7698 hybrid_clock.cc:648] HybridClock initialized: now 1790679239316236 us; error 34 us; skew 500 ppm
Sep 29 10:53:59 dist-test-slave-vqch krb5kdc[7590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679239, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:53:59.318878 7698 init.cc:377] Logged in from keytab as kudu/127.20.80.193@KRBTEST.COM (short username kudu)
I20260929 10:53:59.319347 7698 webserver.cc:460] Webserver started at http://127.20.80.193:34869/ using document root <none> and password file <none>
I20260929 10:53:59.319586 7698 fs_manager.cc:359] Metadata directory not provided
I20260929 10:53:59.319633 7698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:53:59.319780 7698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:53:59.320827 7698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-0/data/instance:
uuid: "af2a392d9eca4661830dfdb6885eb27e"
format_stamp: "Formatted at 2026-09-29 10:53:59 on dist-test-slave-vqch"
server_key: "41c4cdd7aca63bd004b3548cba811169"
server_key_iv: "8dd46070dd43eff4a758d5cca238156b"
server_key_version: "encryptionkey@0"
I20260929 10:53:59.321220 7698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance:
uuid: "af2a392d9eca4661830dfdb6885eb27e"
format_stamp: "Formatted at 2026-09-29 10:53:59 on dist-test-slave-vqch"
server_key: "41c4cdd7aca63bd004b3548cba811169"
server_key_iv: "8dd46070dd43eff4a758d5cca238156b"
server_key_version: "encryptionkey@0"
I20260929 10:53:59.322631 7698 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:53:59.323424 7714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:59.323623 7698 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:53:59.323727 7698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-0/wal
uuid: "af2a392d9eca4661830dfdb6885eb27e"
format_stamp: "Formatted at 2026-09-29 10:53:59 on dist-test-slave-vqch"
server_key: "41c4cdd7aca63bd004b3548cba811169"
server_key_iv: "8dd46070dd43eff4a758d5cca238156b"
server_key_version: "encryptionkey@0"
I20260929 10:53:59.323800 7698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:53:59.338280 7698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:53:59.340708 7698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:53:59.340875 7698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:53:59.341145 7698 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:53:59.341476 7698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:53:59.341529 7698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:59.341572 7698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:53:59.341599 7698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:59.347314 7698 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.193:46869
I20260929 10:53:59.347359 7828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.193:46869 every 8 connection(s)
I20260929 10:53:59.347705 7698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
I20260929 10:53:59.349056 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 7698
I20260929 10:53:59.349182 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance
I20260929 10:53:59.349350 20803 external_mini_cluster.cc:1428] Setting key 6beee7fd868c11fa2e997ea690ab3b43
Sep 29 10:53:59 dist-test-slave-vqch krb5kdc[7590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679239, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.193@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.80.194@KRBTEST.COM; defaulting to no policy
I20260929 10:53:59.358317 7675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:59.348528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:51033 (local address 127.20.80.254:44355)
0929 10:53:59.348577 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:59.348580 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:59.348814 (+ 234us) server_negotiation.cc:407] Connection header received
0929 10:53:59.349505 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:59.349507 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:59.349523 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:59.349597 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:53:59.350741 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.351361 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.352377 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.352744 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.352817 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:59.355191 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:53:59.355192 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:53:59.355193 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:53:59.355204 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:53:59.356853 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:59.357305 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:59.357307 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:59.357307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:59.357373 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:59.357620 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:59.357621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:59.357621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:59.357778 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:53:59.357854 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:59.358104 (+ 250us) server_negotiation.cc:299] Negotiation successful
0929 10:53:59.358214 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 10:53:59.358394 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:59.348449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44355 (local address 127.20.80.193:51033)
0929 10:53:59.348628 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:59.348712 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:59.349394 (+ 682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:59.349654 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:59.349658 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:59.350429 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:53:59.350623 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:59.350629 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.351471 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.351471 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:59.352284 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:59.352286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.352828 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.352829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:59.352928 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:59.353495 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:53:59.353502 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:53:59.355091 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:53:59.356943 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:53:59.356949 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:53:59.356951 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:53:59.357210 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:53:59.357444 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:53:59.357444 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:53:59.357444 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:53:59.357538 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:53:59.357844 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:53:59.357849 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:53:59.357961 (+ 112us) client_negotiation.cc:769] Sending connection context
0929 10:53:59.358035 (+ 74us) client_negotiation.cc:240] Negotiation successful
0929 10:53:59.358193 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":95,"threads_started":1}
I20260929 10:53:59.358934 7829 heartbeater.cc:347] Connected to a master server at 127.20.80.254:44355
I20260929 10:53:59.359048 7829 heartbeater.cc:464] Registering TS with master...
I20260929 10:53:59.359287 7829 heartbeater.cc:511] Master 127.20.80.254:44355 requested a full tablet report, sending...
I20260929 10:53:59.359829 7635 ts_manager.cc:194] Registered new tserver with Master: af2a392d9eca4661830dfdb6885eb27e (127.20.80.193:46869)
I20260929 10:53:59.360491 7635 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.193@KRBTEST.COM'} at 127.20.80.193:51033
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
Entry for principal kudu/127.20.80.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab.
I20260929 10:53:59.375458 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.194:0
--local_ip_for_outbound_sockets=127.20.80.194
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:44355
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:45981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:53:59.456794 7834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:53:59.457067 7834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:53:59.457116 7834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:53:59.458272 7834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:53:59.458312 7834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:53:59.458364 7834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.194
I20260929 10:53:59.459889 7834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:45981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.194.keytab
--principal=kudu/127.20.80.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:44355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.194
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:53:59.460136 7834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:53:59.460369 7834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:53:59.465397 7840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:59.465458 7842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:53:59.465524 7834 server_base.cc:1034] running on GCE node
W20260929 10:53:59.465569 7839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:53:59.465893 7834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:53:59.466223 7834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:53:59.467393 7834 hybrid_clock.cc:648] HybridClock initialized: now 1790679239467370 us; error 38 us; skew 500 ppm
Sep 29 10:53:59 dist-test-slave-vqch krb5kdc[7590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679239, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:53:59.470023 7834 init.cc:377] Logged in from keytab as kudu/127.20.80.194@KRBTEST.COM (short username kudu)
I20260929 10:53:59.470505 7834 webserver.cc:460] Webserver started at http://127.20.80.194:46001/ using document root <none> and password file <none>
I20260929 10:53:59.470753 7834 fs_manager.cc:359] Metadata directory not provided
I20260929 10:53:59.470820 7834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:53:59.470966 7834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:53:59.472041 7834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-1/data/instance:
uuid: "39640dc3aeb24dbea323d2de6efb18cb"
format_stamp: "Formatted at 2026-09-29 10:53:59 on dist-test-slave-vqch"
server_key: "2fea2868961e1529c30a6897e8dfc9ce"
server_key_iv: "6f3d24914fa5683e1654c4960789e9b5"
server_key_version: "encryptionkey@0"
I20260929 10:53:59.472433 7834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance:
uuid: "39640dc3aeb24dbea323d2de6efb18cb"
format_stamp: "Formatted at 2026-09-29 10:53:59 on dist-test-slave-vqch"
server_key: "2fea2868961e1529c30a6897e8dfc9ce"
server_key_iv: "6f3d24914fa5683e1654c4960789e9b5"
server_key_version: "encryptionkey@0"
I20260929 10:53:59.473888 7834 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:53:59.474655 7849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:59.474884 7834 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:53:59.474995 7834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-1/wal
uuid: "39640dc3aeb24dbea323d2de6efb18cb"
format_stamp: "Formatted at 2026-09-29 10:53:59 on dist-test-slave-vqch"
server_key: "2fea2868961e1529c30a6897e8dfc9ce"
server_key_iv: "6f3d24914fa5683e1654c4960789e9b5"
server_key_version: "encryptionkey@0"
I20260929 10:53:59.475072 7834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:53:59.491487 7834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:53:59.493538 7834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:53:59.493707 7834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:53:59.493980 7834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:53:59.494292 7834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:53:59.494345 7834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:59.494381 7834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:53:59.494408 7834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:59.500254 7834 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.194:43541
I20260929 10:53:59.500309 7963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.194:43541 every 8 connection(s)
I20260929 10:53:59.500660 7834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
Sep 29 10:53:59 dist-test-slave-vqch krb5kdc[7590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679239, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.194@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:53:59.509573 7675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:59.501481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:52379 (local address 127.20.80.254:44355)
0929 10:53:59.501567 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:59.501570 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:59.501738 (+ 168us) server_negotiation.cc:407] Connection header received
0929 10:53:59.502239 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:59.502240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:59.502251 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:59.502311 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:53:59.503246 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.503781 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.504786 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.505025 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.505087 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:59.507151 (+ 2064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:53:59.507152 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:53:59.507152 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:53:59.507158 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:53:59.508356 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:59.508693 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:59.508694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:59.508694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:59.508782 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:59.509037 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:59.509038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:59.509038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:59.509151 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:53:59.509204 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:59.509443 (+ 239us) server_negotiation.cc:299] Negotiation successful
0929 10:53:59.509508 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 10:53:59.509725 7966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:59.501441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44355 (local address 127.20.80.194:52379)
0929 10:53:59.501616 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:59.501651 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:59.502125 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:59.502387 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:59.502391 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:59.502963 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:53:59.503151 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:59.503155 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.503864 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.503865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:59.504656 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:59.504658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.505102 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.505103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:59.505210 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:59.505764 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:53:59.505772 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:53:59.507071 (+ 1299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:53:59.508425 (+ 1354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:53:59.508430 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:53:59.508432 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:53:59.508630 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:53:59.508842 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:53:59.508842 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:53:59.508842 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:53:59.508938 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:53:59.509208 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:53:59.509213 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:53:59.509333 (+ 120us) client_negotiation.cc:769] Sending connection context
0929 10:53:59.509417 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 10:53:59.509530 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":99,"threads_started":1}
I20260929 10:53:59.510296 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 7834
I20260929 10:53:59.510289 7964 heartbeater.cc:347] Connected to a master server at 127.20.80.254:44355
I20260929 10:53:59.510396 7964 heartbeater.cc:464] Registering TS with master...
I20260929 10:53:59.510413 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance
I20260929 10:53:59.510592 7964 heartbeater.cc:511] Master 127.20.80.254:44355 requested a full tablet report, sending...
I20260929 10:53:59.510607 20803 external_mini_cluster.cc:1428] Setting key 05c00242bc343f03e92042bdc2f5e3e4
I20260929 10:53:59.511056 7634 ts_manager.cc:194] Registered new tserver with Master: 39640dc3aeb24dbea323d2de6efb18cb (127.20.80.194:43541)
I20260929 10:53:59.511430 7634 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.194@KRBTEST.COM'} at 127.20.80.194:52379
WARNING: no policy specified for kudu/127.20.80.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
Entry for principal kudu/127.20.80.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab.
I20260929 10:53:59.536415 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.195:0
--local_ip_for_outbound_sockets=127.20.80.195
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:44355
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:45981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:53:59.617377 7969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:53:59.617573 7969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:53:59.617596 7969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:53:59.618677 7969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:53:59.618712 7969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:53:59.618737 7969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.195
I20260929 10:53:59.620158 7969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:45981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.195.keytab
--principal=kudu/127.20.80.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:44355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.195
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:53:59.620337 7969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:53:59.620502 7969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:53:59.625823 7974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:59.625854 7975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:53:59.625833 7977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:53:59.625975 7969 server_base.cc:1034] running on GCE node
I20260929 10:53:59.626297 7969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:53:59.626546 7969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:53:59.627754 7969 hybrid_clock.cc:648] HybridClock initialized: now 1790679239627739 us; error 31 us; skew 500 ppm
Sep 29 10:53:59 dist-test-slave-vqch krb5kdc[7590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679239, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:53:59.630415 7969 init.cc:377] Logged in from keytab as kudu/127.20.80.195@KRBTEST.COM (short username kudu)
I20260929 10:53:59.630928 7969 webserver.cc:460] Webserver started at http://127.20.80.195:44543/ using document root <none> and password file <none>
I20260929 10:53:59.631163 7969 fs_manager.cc:359] Metadata directory not provided
I20260929 10:53:59.631232 7969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:53:59.631369 7969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:53:59.632483 7969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-2/data/instance:
uuid: "57712caa76294de9a160dd90a0194d99"
format_stamp: "Formatted at 2026-09-29 10:53:59 on dist-test-slave-vqch"
server_key: "4d73486f22da7ecd02e3c2a91a190172"
server_key_iv: "5129cfb1e305644bb3e9e471f4967f36"
server_key_version: "encryptionkey@0"
I20260929 10:53:59.632885 7969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance:
uuid: "57712caa76294de9a160dd90a0194d99"
format_stamp: "Formatted at 2026-09-29 10:53:59 on dist-test-slave-vqch"
server_key: "4d73486f22da7ecd02e3c2a91a190172"
server_key_iv: "5129cfb1e305644bb3e9e471f4967f36"
server_key_version: "encryptionkey@0"
I20260929 10:53:59.634306 7969 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:53:59.635207 7984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:59.635406 7969 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 10:53:59.635505 7969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-2/wal
uuid: "57712caa76294de9a160dd90a0194d99"
format_stamp: "Formatted at 2026-09-29 10:53:59 on dist-test-slave-vqch"
server_key: "4d73486f22da7ecd02e3c2a91a190172"
server_key_iv: "5129cfb1e305644bb3e9e471f4967f36"
server_key_version: "encryptionkey@0"
I20260929 10:53:59.635593 7969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:53:59.663442 7969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:53:59.665923 7969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:53:59.666091 7969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:53:59.666348 7969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:53:59.666688 7969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:53:59.666744 7969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:59.666788 7969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:53:59.666812 7969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:53:59.672791 7969 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.195:37951
I20260929 10:53:59.672935 8098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.195:37951 every 8 connection(s)
I20260929 10:53:59.673249 7969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
Sep 29 10:53:59 dist-test-slave-vqch krb5kdc[7590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679239, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.195@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:53:59.681527 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 7969
I20260929 10:53:59.681608 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance
I20260929 10:53:59.681821 20803 external_mini_cluster.cc:1428] Setting key 6759624508f054e728c9e88330332b58
I20260929 10:53:59.683295 7675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:59.674125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:35395 (local address 127.20.80.254:44355)
0929 10:53:59.674172 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:59.674174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:59.674361 (+ 187us) server_negotiation.cc:407] Connection header received
0929 10:53:59.674874 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:59.674886 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:59.674902 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:59.674967 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:53:59.676018 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.676532 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.677712 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.677997 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.678091 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:59.680299 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:53:59.680300 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:53:59.680301 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:53:59.680308 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:53:59.681719 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:59.682162 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:59.682163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:59.682164 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:59.682251 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:59.682585 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:59.682586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:59.682586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:59.682725 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:53:59.682790 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:59.683161 (+ 371us) server_negotiation.cc:299] Negotiation successful
0929 10:53:59.683230 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 10:53:59.683450 8101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:59.674036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:44355 (local address 127.20.80.195:35395)
0929 10:53:59.674215 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:59.674257 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:59.674760 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:59.675060 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:59.675064 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:59.675698 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:53:59.675905 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:59.675911 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.676675 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.676676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:59.677603 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:59.677607 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.678120 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.678121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:59.678243 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:59.678739 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:53:59.678747 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:53:59.680185 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:53:59.681804 (+ 1619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:53:59.681812 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:53:59.681814 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:53:59.682081 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:53:59.682385 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:53:59.682386 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:53:59.682387 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:53:59.682513 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:53:59.682889 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:53:59.682894 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:53:59.683033 (+ 139us) client_negotiation.cc:769] Sending connection context
0929 10:53:59.683121 (+ 88us) client_negotiation.cc:240] Negotiation successful
0929 10:53:59.683247 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":117,"threads_started":1}
I20260929 10:53:59.683892 8099 heartbeater.cc:347] Connected to a master server at 127.20.80.254:44355
I20260929 10:53:59.684005 8099 heartbeater.cc:464] Registering TS with master...
I20260929 10:53:59.684190 8099 heartbeater.cc:511] Master 127.20.80.254:44355 requested a full tablet report, sending...
I20260929 10:53:59.684576 7634 ts_manager.cc:194] Registered new tserver with Master: 57712caa76294de9a160dd90a0194d99 (127.20.80.195:37951)
I20260929 10:53:59.684914 7634 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.80.195@KRBTEST.COM'} at 127.20.80.195:35395
I20260929 10:53:59.694077 20803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:53:59.701720 7675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:59.695544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34530 (local address 127.20.80.254:44355)
0929 10:53:59.695619 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:59.695621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:59.695713 (+ 92us) server_negotiation.cc:407] Connection header received
0929 10:53:59.695756 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:59.695756 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:59.695765 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:59.695815 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:53:59.696351 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.696876 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.697631 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.697832 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.697888 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:59.699113 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:53:59.699114 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:53:59.699114 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:53:59.699120 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:53:59.700472 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:59.700820 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:59.700821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:59.700822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:59.700872 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:53:59.701236 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:53:59.701237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:53:59.701237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:53:59.701391 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:53:59.701473 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:59.701597 (+ 124us) server_negotiation.cc:299] Negotiation successful
0929 10:53:59.701656 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 10:53:59.703575 7634 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34530:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:53:59.705206 7634 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:53:59.711900 8112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:59.707843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40634 (local address 127.20.80.193:46869)
0929 10:53:59.708216 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:59.708221 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:59.708229 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:53:59.708260 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:59.708261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:59.708273 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:59.708365 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:53:59.708538 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.709464 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.710838 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.711481 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.711565 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:59.711627 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:59.711776 (+ 149us) server_negotiation.cc:299] Negotiation successful
0929 10:53:59.711816 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":86,"threads_started":1}
I20260929 10:53:59.711889 8111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:59.707767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:46869 (local address 127.0.0.1:40634)
0929 10:53:59.707952 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:59.707996 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:59.708064 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:59.708371 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:59.708373 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:59.708376 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:53:59.708449 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:59.708452 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.709565 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.709566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:59.710749 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:59.710751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.711577 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.711578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:59.711661 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:59.711677 (+ 16us) client_negotiation.cc:769] Sending connection context
0929 10:53:59.711757 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 10:53:59.711814 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":54,"threads_started":1}
I20260929 10:53:59.712298 8114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:59.707963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:43541 (local address 127.0.0.1:58756)
0929 10:53:59.708090 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:59.708098 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:59.708138 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:59.708513 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:59.708514 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:59.708517 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:53:59.708574 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:59.708576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.709982 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.709983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:59.711112 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:59.711115 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.712012 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.712013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:59.712106 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:59.712110 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:53:59.712170 (+ 60us) client_negotiation.cc:240] Negotiation successful
0929 10:53:59.712224 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":83,"threads_started":1}
I20260929 10:53:59.712337 8115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:59.708164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58756 (local address 127.20.80.194:43541)
0929 10:53:59.708346 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:59.708351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:59.708360 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:53:59.708400 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:59.708402 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:59.708416 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:59.708649 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:53:59.708725 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.709874 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.711236 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.711935 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.712010 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:59.712107 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:59.712182 (+ 75us) server_negotiation.cc:299] Negotiation successful
0929 10:53:59.712214 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":116,"threads_started":1}
I20260929 10:53:59.712683 8110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:59.707792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:37951 (local address 127.0.0.1:44422)
0929 10:53:59.708020 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:59.708031 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:59.708082 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:59.708739 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:59.708739 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:59.708742 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:53:59.708779 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:59.708781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.710030 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.710031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:59.711086 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:59.711089 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.712465 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.712466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:59.712526 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:59.712529 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:53:59.712577 (+ 48us) client_negotiation.cc:240] Negotiation successful
0929 10:53:59.712615 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":8,"thread_start_us":90,"threads_started":1}
I20260929 10:53:59.712931 8113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:59.707900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44422 (local address 127.20.80.195:37951)
0929 10:53:59.708508 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:59.708512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:59.708521 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:53:59.708558 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:59.708560 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:59.708576 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:59.708665 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:53:59.708865 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.709942 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.711186 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.711935 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.712010 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:59.712063 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:59.712791 (+ 728us) server_negotiation.cc:299] Negotiation successful
0929 10:53:59.712841 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":75,"threads_started":1}
I20260929 10:53:59.713040 7898 tablet_service.cc:1467] Processing CreateTablet for tablet d2f3d778fbdd4780b0210c866bc0c2e7 (DEFAULT_TABLE table=test-table [id=eeab6f89fc3143b69a4b89bc05d0354c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:53:59.713189 7762 tablet_service.cc:1467] Processing CreateTablet for tablet d2f3d778fbdd4780b0210c866bc0c2e7 (DEFAULT_TABLE table=test-table [id=eeab6f89fc3143b69a4b89bc05d0354c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:53:59.713438 7898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2f3d778fbdd4780b0210c866bc0c2e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:53:59.713481 7762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2f3d778fbdd4780b0210c866bc0c2e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:53:59.716841 8117 tablet_bootstrap.cc:492] T d2f3d778fbdd4780b0210c866bc0c2e7 P 39640dc3aeb24dbea323d2de6efb18cb: Bootstrap starting.
I20260929 10:53:59.717660 8117 tablet_bootstrap.cc:654] T d2f3d778fbdd4780b0210c866bc0c2e7 P 39640dc3aeb24dbea323d2de6efb18cb: Neither blocks nor log segments found. Creating new log.
I20260929 10:53:59.717545 8033 tablet_service.cc:1467] Processing CreateTablet for tablet d2f3d778fbdd4780b0210c866bc0c2e7 (DEFAULT_TABLE table=test-table [id=eeab6f89fc3143b69a4b89bc05d0354c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:53:59.717913 8033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2f3d778fbdd4780b0210c866bc0c2e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:53:59.718248 8117 log.cc:824] T d2f3d778fbdd4780b0210c866bc0c2e7 P 39640dc3aeb24dbea323d2de6efb18cb: Log is configured to *not* fsync() on all Append() calls
I20260929 10:53:59.718603 8116 tablet_bootstrap.cc:492] T d2f3d778fbdd4780b0210c866bc0c2e7 P af2a392d9eca4661830dfdb6885eb27e: Bootstrap starting.
I20260929 10:53:59.719158 8117 tablet_bootstrap.cc:492] T d2f3d778fbdd4780b0210c866bc0c2e7 P 39640dc3aeb24dbea323d2de6efb18cb: No bootstrap required, opened a new log
I20260929 10:53:59.719241 8117 ts_tablet_manager.cc:1397] T d2f3d778fbdd4780b0210c866bc0c2e7 P 39640dc3aeb24dbea323d2de6efb18cb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:53:59.719250 8116 tablet_bootstrap.cc:654] T d2f3d778fbdd4780b0210c866bc0c2e7 P af2a392d9eca4661830dfdb6885eb27e: Neither blocks nor log segments found. Creating new log.
I20260929 10:53:59.719848 8116 log.cc:824] T d2f3d778fbdd4780b0210c866bc0c2e7 P af2a392d9eca4661830dfdb6885eb27e: Log is configured to *not* fsync() on all Append() calls
I20260929 10:53:59.720695 8116 tablet_bootstrap.cc:492] T d2f3d778fbdd4780b0210c866bc0c2e7 P af2a392d9eca4661830dfdb6885eb27e: No bootstrap required, opened a new log
I20260929 10:53:59.720769 8116 ts_tablet_manager.cc:1397] T d2f3d778fbdd4780b0210c866bc0c2e7 P af2a392d9eca4661830dfdb6885eb27e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:53:59.721181 8117 raft_consensus.cc:348] T d2f3d778fbdd4780b0210c866bc0c2e7 P 39640dc3aeb24dbea323d2de6efb18cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2a392d9eca4661830dfdb6885eb27e" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 46869 } } peers { permanent_uuid: "57712caa76294de9a160dd90a0194d99" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 37951 } } peers { permanent_uuid: "39640dc3aeb24dbea323d2de6efb18cb" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 43541 } }
I20260929 10:53:59.721436 8120 tablet_bootstrap.cc:492] T d2f3d778fbdd4780b0210c866bc0c2e7 P 57712caa76294de9a160dd90a0194d99: Bootstrap starting.
I20260929 10:53:59.721655 8117 raft_consensus.cc:374] T d2f3d778fbdd4780b0210c866bc0c2e7 P 39640dc3aeb24dbea323d2de6efb18cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:53:59.721740 8117 raft_consensus.cc:729] T d2f3d778fbdd4780b0210c866bc0c2e7 P 39640dc3aeb24dbea323d2de6efb18cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39640dc3aeb24dbea323d2de6efb18cb, State: Initialized, Role: FOLLOWER
I20260929 10:53:59.721939 8117 consensus_queue.cc:260] T d2f3d778fbdd4780b0210c866bc0c2e7 P 39640dc3aeb24dbea323d2de6efb18cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2a392d9eca4661830dfdb6885eb27e" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 46869 } } peers { permanent_uuid: "57712caa76294de9a160dd90a0194d99" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 37951 } } peers { permanent_uuid: "39640dc3aeb24dbea323d2de6efb18cb" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 43541 } }
I20260929 10:53:59.722182 8120 tablet_bootstrap.cc:654] T d2f3d778fbdd4780b0210c866bc0c2e7 P 57712caa76294de9a160dd90a0194d99: Neither blocks nor log segments found. Creating new log.
I20260929 10:53:59.722339 7964 heartbeater.cc:503] Master 127.20.80.254:44355 was elected leader, sending a full tablet report...
I20260929 10:53:59.722684 8116 raft_consensus.cc:348] T d2f3d778fbdd4780b0210c866bc0c2e7 P af2a392d9eca4661830dfdb6885eb27e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2a392d9eca4661830dfdb6885eb27e" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 46869 } } peers { permanent_uuid: "57712caa76294de9a160dd90a0194d99" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 37951 } } peers { permanent_uuid: "39640dc3aeb24dbea323d2de6efb18cb" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 43541 } }
I20260929 10:53:59.722808 8116 raft_consensus.cc:374] T d2f3d778fbdd4780b0210c866bc0c2e7 P af2a392d9eca4661830dfdb6885eb27e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:53:59.722828 8116 raft_consensus.cc:729] T d2f3d778fbdd4780b0210c866bc0c2e7 P af2a392d9eca4661830dfdb6885eb27e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af2a392d9eca4661830dfdb6885eb27e, State: Initialized, Role: FOLLOWER
I20260929 10:53:59.722827 8120 log.cc:824] T d2f3d778fbdd4780b0210c866bc0c2e7 P 57712caa76294de9a160dd90a0194d99: Log is configured to *not* fsync() on all Append() calls
I20260929 10:53:59.722940 8116 consensus_queue.cc:260] T d2f3d778fbdd4780b0210c866bc0c2e7 P af2a392d9eca4661830dfdb6885eb27e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2a392d9eca4661830dfdb6885eb27e" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 46869 } } peers { permanent_uuid: "57712caa76294de9a160dd90a0194d99" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 37951 } } peers { permanent_uuid: "39640dc3aeb24dbea323d2de6efb18cb" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 43541 } }
I20260929 10:53:59.723222 8116 ts_tablet_manager.cc:1428] T d2f3d778fbdd4780b0210c866bc0c2e7 P af2a392d9eca4661830dfdb6885eb27e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:53:59.722334 8117 ts_tablet_manager.cc:1428] T d2f3d778fbdd4780b0210c866bc0c2e7 P 39640dc3aeb24dbea323d2de6efb18cb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 10:53:59.723402 7829 heartbeater.cc:503] Master 127.20.80.254:44355 was elected leader, sending a full tablet report...
I20260929 10:53:59.723937 8120 tablet_bootstrap.cc:492] T d2f3d778fbdd4780b0210c866bc0c2e7 P 57712caa76294de9a160dd90a0194d99: No bootstrap required, opened a new log
I20260929 10:53:59.724069 8120 ts_tablet_manager.cc:1397] T d2f3d778fbdd4780b0210c866bc0c2e7 P 57712caa76294de9a160dd90a0194d99: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 10:53:59.725899 8120 raft_consensus.cc:348] T d2f3d778fbdd4780b0210c866bc0c2e7 P 57712caa76294de9a160dd90a0194d99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2a392d9eca4661830dfdb6885eb27e" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 46869 } } peers { permanent_uuid: "57712caa76294de9a160dd90a0194d99" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 37951 } } peers { permanent_uuid: "39640dc3aeb24dbea323d2de6efb18cb" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 43541 } }
I20260929 10:53:59.726061 8120 raft_consensus.cc:374] T d2f3d778fbdd4780b0210c866bc0c2e7 P 57712caa76294de9a160dd90a0194d99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:53:59.726109 8120 raft_consensus.cc:729] T d2f3d778fbdd4780b0210c866bc0c2e7 P 57712caa76294de9a160dd90a0194d99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57712caa76294de9a160dd90a0194d99, State: Initialized, Role: FOLLOWER
I20260929 10:53:59.726250 8120 consensus_queue.cc:260] T d2f3d778fbdd4780b0210c866bc0c2e7 P 57712caa76294de9a160dd90a0194d99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2a392d9eca4661830dfdb6885eb27e" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 46869 } } peers { permanent_uuid: "57712caa76294de9a160dd90a0194d99" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 37951 } } peers { permanent_uuid: "39640dc3aeb24dbea323d2de6efb18cb" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 43541 } }
I20260929 10:53:59.726511 8120 ts_tablet_manager.cc:1428] T d2f3d778fbdd4780b0210c866bc0c2e7 P 57712caa76294de9a160dd90a0194d99: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260929 10:53:59.726603 8099 heartbeater.cc:503] Master 127.20.80.254:44355 was elected leader, sending a full tablet report...
W20260929 10:53:59.751283 7965 tablet.cc:2369] T d2f3d778fbdd4780b0210c866bc0c2e7 P 39640dc3aeb24dbea323d2de6efb18cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:53:59.820035 8123 raft_consensus.cc:482] T d2f3d778fbdd4780b0210c866bc0c2e7 P af2a392d9eca4661830dfdb6885eb27e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:53:59.820195 8123 raft_consensus.cc:504] T d2f3d778fbdd4780b0210c866bc0c2e7 P af2a392d9eca4661830dfdb6885eb27e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2a392d9eca4661830dfdb6885eb27e" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 46869 } } peers { permanent_uuid: "57712caa76294de9a160dd90a0194d99" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 37951 } } peers { permanent_uuid: "39640dc3aeb24dbea323d2de6efb18cb" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 43541 } }
I20260929 10:53:59.820585 8123 leader_election.cc:288] T d2f3d778fbdd4780b0210c866bc0c2e7 P af2a392d9eca4661830dfdb6885eb27e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57712caa76294de9a160dd90a0194d99 (127.20.80.195:37951), 39640dc3aeb24dbea323d2de6efb18cb (127.20.80.194:43541)
I20260929 10:53:59.824070 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:59.820809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:37951 (local address 127.20.80.193:42845)
0929 10:53:59.820869 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:59.820884 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:59.820936 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:59.821151 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:59.821151 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:59.821154 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:53:59.821237 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:59.821239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.822228 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.822229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:59.823074 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:59.823107 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.823798 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.823799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:59.823891 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:59.823895 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:53:59.823951 (+ 56us) client_negotiation.cc:240] Negotiation successful
0929 10:53:59.823992 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":30}
I20260929 10:53:59.824090 8113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:59.820918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:42845 (local address 127.20.80.195:37951)
0929 10:53:59.820970 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:59.820972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:59.820981 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:53:59.821051 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:59.821052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:59.821068 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:59.821121 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:53:59.821340 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.822134 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.823196 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.823722 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.823781 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:59.823788 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:59.823964 (+ 176us) server_negotiation.cc:299] Negotiation successful
0929 10:53:59.824006 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 10:53:59.824201 8125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:59.820817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:43541 (local address 127.20.80.193:48683)
0929 10:53:59.821087 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:59.821095 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 10:53:59.821172 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:53:59.821308 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:53:59.821309 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:53:59.821311 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:53:59.821372 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:59.821373 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.822226 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.822227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:59.823136 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:53:59.823139 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.823912 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.823913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:53:59.824000 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:59.824003 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:53:59.824062 (+ 59us) client_negotiation.cc:240] Negotiation successful
0929 10:53:59.824099 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":104,"thread_start_us":132,"threads_started":1}
I20260929 10:53:59.824275 8115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:59.820891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:48683 (local address 127.20.80.194:43541)
0929 10:53:59.820939 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:59.820941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:59.821182 (+ 241us) server_negotiation.cc:407] Connection header received
0929 10:53:59.821231 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:59.821232 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:59.821243 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:59.821283 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:53:59.821436 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.822114 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.823242 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.823827 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.823900 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:59.823910 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:59.824087 (+ 177us) server_negotiation.cc:299] Negotiation successful
0929 10:53:59.824136 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 10:53:59.824657 7918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f3d778fbdd4780b0210c866bc0c2e7" candidate_uuid: "af2a392d9eca4661830dfdb6885eb27e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39640dc3aeb24dbea323d2de6efb18cb" is_pre_election: true
I20260929 10:53:59.824791 7918 raft_consensus.cc:2454] T d2f3d778fbdd4780b0210c866bc0c2e7 P 39640dc3aeb24dbea323d2de6efb18cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af2a392d9eca4661830dfdb6885eb27e in term 0.
I20260929 10:53:59.824790 8053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f3d778fbdd4780b0210c866bc0c2e7" candidate_uuid: "af2a392d9eca4661830dfdb6885eb27e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57712caa76294de9a160dd90a0194d99" is_pre_election: true
I20260929 10:53:59.824936 8053 raft_consensus.cc:2454] T d2f3d778fbdd4780b0210c866bc0c2e7 P 57712caa76294de9a160dd90a0194d99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af2a392d9eca4661830dfdb6885eb27e in term 0.
I20260929 10:53:59.825049 7717 leader_election.cc:302] T d2f3d778fbdd4780b0210c866bc0c2e7 P af2a392d9eca4661830dfdb6885eb27e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39640dc3aeb24dbea323d2de6efb18cb, af2a392d9eca4661830dfdb6885eb27e; no voters:
I20260929 10:53:59.825186 8123 raft_consensus.cc:2781] T d2f3d778fbdd4780b0210c866bc0c2e7 P af2a392d9eca4661830dfdb6885eb27e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:53:59.825251 8123 raft_consensus.cc:482] T d2f3d778fbdd4780b0210c866bc0c2e7 P af2a392d9eca4661830dfdb6885eb27e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:53:59.825285 8123 raft_consensus.cc:3037] T d2f3d778fbdd4780b0210c866bc0c2e7 P af2a392d9eca4661830dfdb6885eb27e [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:53:59.825954 8123 raft_consensus.cc:504] T d2f3d778fbdd4780b0210c866bc0c2e7 P af2a392d9eca4661830dfdb6885eb27e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2a392d9eca4661830dfdb6885eb27e" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 46869 } } peers { permanent_uuid: "57712caa76294de9a160dd90a0194d99" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 37951 } } peers { permanent_uuid: "39640dc3aeb24dbea323d2de6efb18cb" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 43541 } }
I20260929 10:53:59.826097 8123 leader_election.cc:288] T d2f3d778fbdd4780b0210c866bc0c2e7 P af2a392d9eca4661830dfdb6885eb27e [CANDIDATE]: Term 1 election: Requested vote from peers 57712caa76294de9a160dd90a0194d99 (127.20.80.195:37951), 39640dc3aeb24dbea323d2de6efb18cb (127.20.80.194:43541)
I20260929 10:53:59.826264 8053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f3d778fbdd4780b0210c866bc0c2e7" candidate_uuid: "af2a392d9eca4661830dfdb6885eb27e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57712caa76294de9a160dd90a0194d99"
I20260929 10:53:59.826306 7918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f3d778fbdd4780b0210c866bc0c2e7" candidate_uuid: "af2a392d9eca4661830dfdb6885eb27e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39640dc3aeb24dbea323d2de6efb18cb"
I20260929 10:53:59.826352 8053 raft_consensus.cc:3037] T d2f3d778fbdd4780b0210c866bc0c2e7 P 57712caa76294de9a160dd90a0194d99 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:53:59.826370 7918 raft_consensus.cc:3037] T d2f3d778fbdd4780b0210c866bc0c2e7 P 39640dc3aeb24dbea323d2de6efb18cb [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:53:59.827278 8053 raft_consensus.cc:2454] T d2f3d778fbdd4780b0210c866bc0c2e7 P 57712caa76294de9a160dd90a0194d99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af2a392d9eca4661830dfdb6885eb27e in term 1.
I20260929 10:53:59.827278 7918 raft_consensus.cc:2454] T d2f3d778fbdd4780b0210c866bc0c2e7 P 39640dc3aeb24dbea323d2de6efb18cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af2a392d9eca4661830dfdb6885eb27e in term 1.
I20260929 10:53:59.827476 7715 leader_election.cc:302] T d2f3d778fbdd4780b0210c866bc0c2e7 P af2a392d9eca4661830dfdb6885eb27e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57712caa76294de9a160dd90a0194d99, af2a392d9eca4661830dfdb6885eb27e; no voters:
I20260929 10:53:59.827567 8123 raft_consensus.cc:2781] T d2f3d778fbdd4780b0210c866bc0c2e7 P af2a392d9eca4661830dfdb6885eb27e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:53:59.827690 8123 raft_consensus.cc:686] T d2f3d778fbdd4780b0210c866bc0c2e7 P af2a392d9eca4661830dfdb6885eb27e [term 1 LEADER]: Becoming Leader. State: Replica: af2a392d9eca4661830dfdb6885eb27e, State: Running, Role: LEADER
I20260929 10:53:59.827801 8123 consensus_queue.cc:237] T d2f3d778fbdd4780b0210c866bc0c2e7 P af2a392d9eca4661830dfdb6885eb27e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2a392d9eca4661830dfdb6885eb27e" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 46869 } } peers { permanent_uuid: "57712caa76294de9a160dd90a0194d99" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 37951 } } peers { permanent_uuid: "39640dc3aeb24dbea323d2de6efb18cb" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 43541 } }
I20260929 10:53:59.828671 7635 catalog_manager.cc:5495] T d2f3d778fbdd4780b0210c866bc0c2e7 P af2a392d9eca4661830dfdb6885eb27e reported cstate change: term changed from 0 to 1, leader changed from <none> to af2a392d9eca4661830dfdb6885eb27e (127.20.80.193). New cstate: current_term: 1 leader_uuid: "af2a392d9eca4661830dfdb6885eb27e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af2a392d9eca4661830dfdb6885eb27e" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 46869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57712caa76294de9a160dd90a0194d99" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 37951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39640dc3aeb24dbea323d2de6efb18cb" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 43541 } health_report { overall_health: UNKNOWN } } }
W20260929 10:53:59.848467 7830 tablet.cc:2369] T d2f3d778fbdd4780b0210c866bc0c2e7 P af2a392d9eca4661830dfdb6885eb27e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:53:59.856578 8112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:59.854397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40650 (local address 127.20.80.193:46869)
0929 10:53:59.854456 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0929 10:53:59.854459 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:53:59.854468 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:53:59.854510 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:53:59.854511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:53:59.854529 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:53:59.854583 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:53:59.854702 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.855160 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.855872 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:53:59.856050 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:53:59.856098 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:53:59.856251 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:53:59.856342 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:53:59.856402 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 10:53:59.856489 (+ 87us) server_negotiation.cc:299] Negotiation successful
0929 10:53:59.856506 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260929 10:53:59.858462 7918 raft_consensus.cc:1261] T d2f3d778fbdd4780b0210c866bc0c2e7 P 39640dc3aeb24dbea323d2de6efb18cb [term 1 FOLLOWER]: Refusing update from remote peer af2a392d9eca4661830dfdb6885eb27e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:53:59.858480 8053 raft_consensus.cc:1261] T d2f3d778fbdd4780b0210c866bc0c2e7 P 57712caa76294de9a160dd90a0194d99 [term 1 FOLLOWER]: Refusing update from remote peer af2a392d9eca4661830dfdb6885eb27e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:53:59.858872 8123 consensus_queue.cc:1035] T d2f3d778fbdd4780b0210c866bc0c2e7 P af2a392d9eca4661830dfdb6885eb27e [LEADER]: Connected to new peer: Peer: permanent_uuid: "39640dc3aeb24dbea323d2de6efb18cb" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 43541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:53:59.859038 8126 consensus_queue.cc:1035] T d2f3d778fbdd4780b0210c866bc0c2e7 P af2a392d9eca4661830dfdb6885eb27e [LEADER]: Connected to new peer: Peer: permanent_uuid: "57712caa76294de9a160dd90a0194d99" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 37951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:53:59.861337 8129 mvcc.cc:205] Tried to move back new op lower bound from 7334622166457360384 to 7334622166335250432. Current Snapshot: MvccSnapshot[applied={T|T < 7334622166457360384}]
I20260929 10:53:59.862157 8131 mvcc.cc:205] Tried to move back new op lower bound from 7334622166457360384 to 7334622166335250432. Current Snapshot: MvccSnapshot[applied={T|T < 7334622166457360384 or (T in {7334622166457360384})}]
I20260929 10:53:59.862489 8130 mvcc.cc:205] Tried to move back new op lower bound from 7334622166457360384 to 7334622166335250432. Current Snapshot: MvccSnapshot[applied={T|T < 7334622166457360384}]
I20260929 10:53:59.863811 7635 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:53:59.863909 7635 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:53:59.864643 7635 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 15732e87f56a4f6dafe7bbd5875f9719: Sending DeleteTablet for 3 replicas of tablet d2f3d778fbdd4780b0210c866bc0c2e7
I20260929 10:53:59.865008 8033 tablet_service.cc:1514] Processing DeleteTablet for tablet d2f3d778fbdd4780b0210c866bc0c2e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:53:59 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:44422
I20260929 10:53:59.865046 7898 tablet_service.cc:1514] Processing DeleteTablet for tablet d2f3d778fbdd4780b0210c866bc0c2e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:53:59 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:58756
I20260929 10:53:59.865073 7762 tablet_service.cc:1514] Processing DeleteTablet for tablet d2f3d778fbdd4780b0210c866bc0c2e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:53:59 UTC) from {username='kudu', principal='kudu/127.20.80.254@KRBTEST.COM'} at 127.0.0.1:40634
I20260929 10:53:59.865375 8138 tablet_replica.cc:330] T d2f3d778fbdd4780b0210c866bc0c2e7 P 57712caa76294de9a160dd90a0194d99: stopping tablet replica
I20260929 10:53:59.865415 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 7698
I20260929 10:53:59.866055 8138 raft_consensus.cc:2229] T d2f3d778fbdd4780b0210c866bc0c2e7 P 57712caa76294de9a160dd90a0194d99 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:53:59.866209 8138 raft_consensus.cc:2258] T d2f3d778fbdd4780b0210c866bc0c2e7 P 57712caa76294de9a160dd90a0194d99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:53:59.867267 8139 tablet_replica.cc:330] T d2f3d778fbdd4780b0210c866bc0c2e7 P 39640dc3aeb24dbea323d2de6efb18cb: stopping tablet replica
I20260929 10:53:59.867379 8139 raft_consensus.cc:2229] T d2f3d778fbdd4780b0210c866bc0c2e7 P 39640dc3aeb24dbea323d2de6efb18cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:53:59.867491 8139 raft_consensus.cc:2258] T d2f3d778fbdd4780b0210c866bc0c2e7 P 39640dc3aeb24dbea323d2de6efb18cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:53:59.867812 8138 ts_tablet_manager.cc:1905] T d2f3d778fbdd4780b0210c866bc0c2e7 P 57712caa76294de9a160dd90a0194d99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:53:59.867939 8139 ts_tablet_manager.cc:1905] T d2f3d778fbdd4780b0210c866bc0c2e7 P 39640dc3aeb24dbea323d2de6efb18cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:53:59.869896 8139 ts_tablet_manager.cc:1918] T d2f3d778fbdd4780b0210c866bc0c2e7 P 39640dc3aeb24dbea323d2de6efb18cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:53:59.869915 8138 ts_tablet_manager.cc:1918] T d2f3d778fbdd4780b0210c866bc0c2e7 P 57712caa76294de9a160dd90a0194d99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:53:59.869974 8138 log.cc:1196] T d2f3d778fbdd4780b0210c866bc0c2e7 P 57712caa76294de9a160dd90a0194d99: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-2/wal/wals/d2f3d778fbdd4780b0210c866bc0c2e7
I20260929 10:53:59.870026 8139 log.cc:1196] T d2f3d778fbdd4780b0210c866bc0c2e7 P 39640dc3aeb24dbea323d2de6efb18cb: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790678996126930-20803-0/minicluster-data/ts-1/wal/wals/d2f3d778fbdd4780b0210c866bc0c2e7
I20260929 10:53:59.870290 8138 ts_tablet_manager.cc:1939] T d2f3d778fbdd4780b0210c866bc0c2e7 P 57712caa76294de9a160dd90a0194d99: Deleting consensus metadata
I20260929 10:53:59.870373 8139 ts_tablet_manager.cc:1939] T d2f3d778fbdd4780b0210c866bc0c2e7 P 39640dc3aeb24dbea323d2de6efb18cb: Deleting consensus metadata
I20260929 10:53:59.870713 7619 catalog_manager.cc:4841] TS 57712caa76294de9a160dd90a0194d99 (127.20.80.195:37951): tablet d2f3d778fbdd4780b0210c866bc0c2e7 (table test-table [id=eeab6f89fc3143b69a4b89bc05d0354c]) successfully deleted
W20260929 10:53:59.871816 7621 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.80.193:46869 (error 108)
I20260929 10:53:59.872062 7621 catalog_manager.cc:4841] TS 39640dc3aeb24dbea323d2de6efb18cb (127.20.80.194:43541): tablet d2f3d778fbdd4780b0210c866bc0c2e7 (table test-table [id=eeab6f89fc3143b69a4b89bc05d0354c]) successfully deleted
I20260929 10:53:59.872402 8110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:53:59.872235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:46869 (local address 127.0.0.1:40658)
0929 10:53:59.872295 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0929 10:53:59.872365 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.193:46869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260929 10:53:59.872493 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 7834
W20260929 10:53:59.872555 7621 catalog_manager.cc:4584] TS af2a392d9eca4661830dfdb6885eb27e (127.20.80.193:46869): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d2f3d778fbdd4780b0210c866bc0c2e7: Network error: Client connection negotiation failed: client connection to 127.20.80.193:46869: connect: Connection refused (error 111)
I20260929 10:53:59.877928 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 7969
I20260929 10:53:59.882243 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 7603
2026-09-29T10:53:59Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2073 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-29T10:53:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:53:59Z Disabled control of system clock
I20260929 10:53:59.909651 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:43629
--webserver_interface=127.20.80.254
--webserver_port=0
--builtin_ntp_servers=127.20.80.212:36785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:43629
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 10:53:59.994504 8146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:53:59.994763 8146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:53:59.994802 8146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:53:59.995941 8146 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:53:59.996011 8146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:53:59.996031 8146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:53:59.996050 8146 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:53:59.996083 8146 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:53:59.997443 8146 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:36785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:43629
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:43629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:53:59.997723 8146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:53:59.998015 8146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:54:00.002871 8152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:00.002895 8154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:00.002888 8151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:00.003085 8146 server_base.cc:1034] running on GCE node
I20260929 10:54:00.003396 8146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:54:00.003687 8146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:54:00.004874 8146 hybrid_clock.cc:648] HybridClock initialized: now 1790679240004834 us; error 54 us; skew 500 ppm
I20260929 10:54:00.006114 8146 webserver.cc:460] Webserver started at http://127.20.80.254:46559/ using document root <none> and password file <none>
I20260929 10:54:00.006388 8146 fs_manager.cc:359] Metadata directory not provided
I20260929 10:54:00.006462 8146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:54:00.006618 8146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:54:00.007807 8146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "2864e06b3c6f4978ba659d46923fbe00"
format_stamp: "Formatted at 2026-09-29 10:54:00 on dist-test-slave-vqch"
server_key: "da72f6d73981450a90be2fbd24fc2f4b"
server_key_iv: "e46d4c0aeb099589c0f3ea4f32cadde4"
server_key_version: "encryptionkey@0"
I20260929 10:54:00.008189 8146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "2864e06b3c6f4978ba659d46923fbe00"
format_stamp: "Formatted at 2026-09-29 10:54:00 on dist-test-slave-vqch"
server_key: "da72f6d73981450a90be2fbd24fc2f4b"
server_key_iv: "e46d4c0aeb099589c0f3ea4f32cadde4"
server_key_version: "encryptionkey@0"
I20260929 10:54:00.010172 8146 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260929 10:54:00.012164 8160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:00.012336 8146 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 10:54:00.012487 8146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "2864e06b3c6f4978ba659d46923fbe00"
format_stamp: "Formatted at 2026-09-29 10:54:00 on dist-test-slave-vqch"
server_key: "da72f6d73981450a90be2fbd24fc2f4b"
server_key_iv: "e46d4c0aeb099589c0f3ea4f32cadde4"
server_key_version: "encryptionkey@0"
I20260929 10:54:00.012569 8146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:54:00.027397 8146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:54:00.027730 8146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:54:00.027837 8146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:54:00.031899 8146 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:43629
I20260929 10:54:00.031956 8212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:43629 every 8 connection(s)
I20260929 10:54:00.032305 8146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:54:00.033385 8213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:54:00.033748 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 8146
I20260929 10:54:00.033835 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:54:00.034047 20803 external_mini_cluster.cc:1428] Setting key f058dcfd13ab6f20ba9405970ed60561
I20260929 10:54:00.036149 8213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2864e06b3c6f4978ba659d46923fbe00: Bootstrap starting.
I20260929 10:54:00.036788 8213 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2864e06b3c6f4978ba659d46923fbe00: Neither blocks nor log segments found. Creating new log.
I20260929 10:54:00.037113 8213 log.cc:824] T 00000000000000000000000000000000 P 2864e06b3c6f4978ba659d46923fbe00: Log is configured to *not* fsync() on all Append() calls
I20260929 10:54:00.037863 8213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2864e06b3c6f4978ba659d46923fbe00: No bootstrap required, opened a new log
I20260929 10:54:00.039637 8216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:00.034595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56016 (local address 127.20.80.254:43629)
0929 10:54:00.034771 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:00.034779 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:00.034803 (+ 24us) server_negotiation.cc:407] Connection header received
0929 10:54:00.035209 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:00.035216 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:00.035274 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:00.035388 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:00.035703 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.036643 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.037729 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.038189 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.038288 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:00.039000 (+ 712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:00.039003 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:00.039004 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:54:00.039161 (+ 157us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:00.039193 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:54:00.039203 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:00.039256 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:00.039351 (+ 95us) server_negotiation.cc:299] Negotiation successful
0929 10:54:00.039435 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":84,"threads_started":1}
I20260929 10:54:00.040184 8213 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2864e06b3c6f4978ba659d46923fbe00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2864e06b3c6f4978ba659d46923fbe00" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 43629 } }
I20260929 10:54:00.040350 8213 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2864e06b3c6f4978ba659d46923fbe00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:54:00.040419 8213 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2864e06b3c6f4978ba659d46923fbe00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2864e06b3c6f4978ba659d46923fbe00, State: Initialized, Role: FOLLOWER
I20260929 10:54:00.040570 8213 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2864e06b3c6f4978ba659d46923fbe00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2864e06b3c6f4978ba659d46923fbe00" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 43629 } }
I20260929 10:54:00.040668 8213 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2864e06b3c6f4978ba659d46923fbe00 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:54:00.040711 8213 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2864e06b3c6f4978ba659d46923fbe00 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:54:00.040761 8213 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2864e06b3c6f4978ba659d46923fbe00 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:54:00.041364 8213 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2864e06b3c6f4978ba659d46923fbe00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2864e06b3c6f4978ba659d46923fbe00" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 43629 } }
I20260929 10:54:00.041482 8213 leader_election.cc:302] T 00000000000000000000000000000000 P 2864e06b3c6f4978ba659d46923fbe00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2864e06b3c6f4978ba659d46923fbe00; no voters:
I20260929 10:54:00.041667 8213 leader_election.cc:288] T 00000000000000000000000000000000 P 2864e06b3c6f4978ba659d46923fbe00 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:54:00.041716 8218 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2864e06b3c6f4978ba659d46923fbe00 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:54:00.041867 8218 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2864e06b3c6f4978ba659d46923fbe00 [term 1 LEADER]: Becoming Leader. State: Replica: 2864e06b3c6f4978ba659d46923fbe00, State: Running, Role: LEADER
I20260929 10:54:00.041944 8213 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2864e06b3c6f4978ba659d46923fbe00 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:54:00.041953 8218 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2864e06b3c6f4978ba659d46923fbe00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2864e06b3c6f4978ba659d46923fbe00" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 43629 } }
I20260929 10:54:00.042404 8220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2864e06b3c6f4978ba659d46923fbe00 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2864e06b3c6f4978ba659d46923fbe00. Latest consensus state: current_term: 1 leader_uuid: "2864e06b3c6f4978ba659d46923fbe00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2864e06b3c6f4978ba659d46923fbe00" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 43629 } } }
I20260929 10:54:00.042690 8220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2864e06b3c6f4978ba659d46923fbe00 [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:00.042609 8219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2864e06b3c6f4978ba659d46923fbe00 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2864e06b3c6f4978ba659d46923fbe00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2864e06b3c6f4978ba659d46923fbe00" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 43629 } } }
I20260929 10:54:00.042835 8219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2864e06b3c6f4978ba659d46923fbe00 [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:00.043072 8227 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:54:00.043543 8227 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:54:00.045010 8227 catalog_manager.cc:1347] Generated new cluster ID: 93227f19a6804215ab7b9a6fa3e287b5
I20260929 10:54:00.045079 8227 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:54:00.056718 8227 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:54:00.057338 8227 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:54:00.059988 8227 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2864e06b3c6f4978ba659d46923fbe00: Generated new TSK 0
I20260929 10:54:00.060161 8227 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:54:00.063654 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.193:0
--local_ip_for_outbound_sockets=127.20.80.193
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:43629
--builtin_ntp_servers=127.20.80.212:36785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 10:54:00.171303 8237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:54:00.171495 8237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:54:00.171516 8237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:54:00.172726 8237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:54:00.172770 8237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:54:00.172796 8237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.193
I20260929 10:54:00.174149 8237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:36785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.80.193
--webserver_port=0
--tserver_master_addrs=127.20.80.254:43629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.193
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:54:00.174400 8237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:54:00.174708 8237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:54:00.179695 8242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:00.179709 8245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:00.179695 8243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:00.180055 8237 server_base.cc:1034] running on GCE node
I20260929 10:54:00.180266 8237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:54:00.180513 8237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:54:00.181707 8237 hybrid_clock.cc:648] HybridClock initialized: now 1790679240181698 us; error 35 us; skew 500 ppm
I20260929 10:54:00.188889 8237 webserver.cc:460] Webserver started at http://127.20.80.193:33329/ using document root <none> and password file <none>
I20260929 10:54:00.189188 8237 fs_manager.cc:359] Metadata directory not provided
I20260929 10:54:00.189250 8237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:54:00.189379 8237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:54:00.190806 8237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-0/data/instance:
uuid: "282cd0e219024df2962adae8c0ead4b3"
format_stamp: "Formatted at 2026-09-29 10:54:00 on dist-test-slave-vqch"
server_key: "6f93e6a80ca647e1a3a0d03bfd022916"
server_key_iv: "8ebc201b7d62fb60d5bf195ad7cf8154"
server_key_version: "encryptionkey@0"
I20260929 10:54:00.191210 8237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance:
uuid: "282cd0e219024df2962adae8c0ead4b3"
format_stamp: "Formatted at 2026-09-29 10:54:00 on dist-test-slave-vqch"
server_key: "6f93e6a80ca647e1a3a0d03bfd022916"
server_key_iv: "8ebc201b7d62fb60d5bf195ad7cf8154"
server_key_version: "encryptionkey@0"
I20260929 10:54:00.192853 8237 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260929 10:54:00.193809 8251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:00.194021 8237 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:54:00.194116 8237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-0/wal
uuid: "282cd0e219024df2962adae8c0ead4b3"
format_stamp: "Formatted at 2026-09-29 10:54:00 on dist-test-slave-vqch"
server_key: "6f93e6a80ca647e1a3a0d03bfd022916"
server_key_iv: "8ebc201b7d62fb60d5bf195ad7cf8154"
server_key_version: "encryptionkey@0"
I20260929 10:54:00.194190 8237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:54:00.225987 8237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:54:00.226289 8237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:54:00.226397 8237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:54:00.226595 8237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:54:00.226974 8237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:54:00.227010 8237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:00.227056 8237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:54:00.227082 8237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:00.233018 8237 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.193:42181
I20260929 10:54:00.233050 8365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.193:42181 every 8 connection(s)
I20260929 10:54:00.233397 8237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-0/data/info.pb
I20260929 10:54:00.238634 8216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:00.234150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.193:47515 (local address 127.20.80.254:43629)
0929 10:54:00.234224 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:00.234226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:00.234466 (+ 240us) server_negotiation.cc:407] Connection header received
0929 10:54:00.234953 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:00.234954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:00.234972 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:00.235041 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:00.235754 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.236301 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.237361 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.237560 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.237636 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:00.238397 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:00.238398 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:00.238398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:54:00.238402 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:00.238417 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:54:00.238421 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:00.238464 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:00.238549 (+ 85us) server_negotiation.cc:299] Negotiation successful
0929 10:54:00.238573 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 10:54:00.238857 8368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:00.234150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:43629 (local address 127.20.80.193:47515)
0929 10:54:00.234322 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:00.234356 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 10:54:00.234842 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:54:00.235114 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:54:00.235117 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:54:00.235125 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:54:00.235645 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:00.235649 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.236397 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.236398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:00.237281 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:00.237284 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.237644 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.237645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:00.237753 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:00.238268 (+ 515us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:54:00.238268 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:54:00.238296 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:54:00.238318 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:54:00.238477 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:54:00.238481 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:54:00.238483 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 10:54:00.238535 (+ 52us) client_negotiation.cc:240] Negotiation successful
0929 10:54:00.238615 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":79,"threads_started":1}
I20260929 10:54:00.239388 8366 heartbeater.cc:347] Connected to a master server at 127.20.80.254:43629
I20260929 10:54:00.239466 8366 heartbeater.cc:464] Registering TS with master...
I20260929 10:54:00.239656 8366 heartbeater.cc:511] Master 127.20.80.254:43629 requested a full tablet report, sending...
I20260929 10:54:00.240180 8177 ts_manager.cc:194] Registered new tserver with Master: 282cd0e219024df2962adae8c0ead4b3 (127.20.80.193:42181)
I20260929 10:54:00.240805 8177 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.80.193:47515
I20260929 10:54:00.241166 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 8237
I20260929 10:54:00.241269 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-0/wal/instance
I20260929 10:54:00.241474 20803 external_mini_cluster.cc:1428] Setting key 45b9cc82268c6dcb898afa11d728033c
I20260929 10:54:00.242327 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.194:0
--local_ip_for_outbound_sockets=127.20.80.194
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:43629
--builtin_ntp_servers=127.20.80.212:36785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 10:54:00.334412 8369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:54:00.334618 8369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:54:00.334640 8369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:54:00.335711 8369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:54:00.335753 8369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:54:00.335777 8369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.194
I20260929 10:54:00.337034 8369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:36785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.80.194
--webserver_port=0
--tserver_master_addrs=127.20.80.254:43629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.194
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:54:00.337209 8369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:54:00.337440 8369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:54:00.342093 8377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:00.342165 8375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:00.342164 8374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:00.342460 8369 server_base.cc:1034] running on GCE node
I20260929 10:54:00.342702 8369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:54:00.342940 8369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:54:00.344118 8369 hybrid_clock.cc:648] HybridClock initialized: now 1790679240344099 us; error 50 us; skew 500 ppm
I20260929 10:54:00.345384 8369 webserver.cc:460] Webserver started at http://127.20.80.194:39783/ using document root <none> and password file <none>
I20260929 10:54:00.345618 8369 fs_manager.cc:359] Metadata directory not provided
I20260929 10:54:00.345687 8369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:54:00.345839 8369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:54:00.346860 8369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-1/data/instance:
uuid: "3394dcdad00f42f485541aa317c6a5f2"
format_stamp: "Formatted at 2026-09-29 10:54:00 on dist-test-slave-vqch"
server_key: "749d1102432a578660752a9351605154"
server_key_iv: "9ecffecb18dcbf83976c8d2e0301ea0b"
server_key_version: "encryptionkey@0"
I20260929 10:54:00.347214 8369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance:
uuid: "3394dcdad00f42f485541aa317c6a5f2"
format_stamp: "Formatted at 2026-09-29 10:54:00 on dist-test-slave-vqch"
server_key: "749d1102432a578660752a9351605154"
server_key_iv: "9ecffecb18dcbf83976c8d2e0301ea0b"
server_key_version: "encryptionkey@0"
I20260929 10:54:00.348515 8369 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 10:54:00.349401 8383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:00.349568 8369 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:54:00.349663 8369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-1/wal
uuid: "3394dcdad00f42f485541aa317c6a5f2"
format_stamp: "Formatted at 2026-09-29 10:54:00 on dist-test-slave-vqch"
server_key: "749d1102432a578660752a9351605154"
server_key_iv: "9ecffecb18dcbf83976c8d2e0301ea0b"
server_key_version: "encryptionkey@0"
I20260929 10:54:00.349763 8369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:54:00.368212 8369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:54:00.368507 8369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:54:00.368654 8369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:54:00.368873 8369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:54:00.369277 8369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:54:00.369331 8369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:00.369365 8369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:54:00.369392 8369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:00.375062 8369 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.194:34943
I20260929 10:54:00.375110 8496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.194:34943 every 8 connection(s)
I20260929 10:54:00.375424 8369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-1/data/info.pb
I20260929 10:54:00.377290 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 8369
I20260929 10:54:00.377408 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-1/wal/instance
I20260929 10:54:00.377615 20803 external_mini_cluster.cc:1428] Setting key 5eb73b2869007dac4a5f00b97b4a7b7e
I20260929 10:54:00.378707 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.80.195:0
--local_ip_for_outbound_sockets=127.20.80.195
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:43629
--builtin_ntp_servers=127.20.80.212:36785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 10:54:00.381908 8216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:00.376129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.194:44211 (local address 127.20.80.254:43629)
0929 10:54:00.376211 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:00.376213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:00.376462 (+ 249us) server_negotiation.cc:407] Connection header received
0929 10:54:00.376951 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:00.376953 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:00.376964 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:00.377098 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:00.377921 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.378626 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.380202 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.380384 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.380440 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:00.381582 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:00.381583 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:00.381583 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:54:00.381588 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:00.381652 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:54:00.381656 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:00.381721 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:00.381805 (+ 84us) server_negotiation.cc:299] Negotiation successful
0929 10:54:00.381831 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 10:54:00.382135 8499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:00.376111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:43629 (local address 127.20.80.194:44211)
0929 10:54:00.376316 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:00.376357 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 10:54:00.376826 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:54:00.377166 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:54:00.377170 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:54:00.377178 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:54:00.377790 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:00.377796 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.378745 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.378746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:00.380101 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:00.380105 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.380473 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.380474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:00.380635 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:00.381360 (+ 725us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:54:00.381361 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:54:00.381430 (+ 69us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:54:00.381462 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:54:00.381723 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:54:00.381731 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:54:00.381735 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:54:00.381799 (+ 64us) client_negotiation.cc:240] Negotiation successful
0929 10:54:00.381903 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":98,"threads_started":1}
I20260929 10:54:00.382692 8497 heartbeater.cc:347] Connected to a master server at 127.20.80.254:43629
I20260929 10:54:00.382807 8497 heartbeater.cc:464] Registering TS with master...
I20260929 10:54:00.383023 8497 heartbeater.cc:511] Master 127.20.80.254:43629 requested a full tablet report, sending...
I20260929 10:54:00.383421 8177 ts_manager.cc:194] Registered new tserver with Master: 3394dcdad00f42f485541aa317c6a5f2 (127.20.80.194:34943)
I20260929 10:54:00.383797 8177 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.80.194:44211
W20260929 10:54:00.463287 8500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:54:00.463502 8500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:54:00.463526 8500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:54:00.464802 8500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:54:00.464843 8500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:54:00.464870 8500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.80.195
I20260929 10:54:00.466408 8500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:36785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.80.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.80.195
--webserver_port=0
--tserver_master_addrs=127.20.80.254:43629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.80.195
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:54:00.466676 8500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:54:00.467005 8500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:54:00.472003 8508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:00.472172 8500 server_base.cc:1034] running on GCE node
W20260929 10:54:00.472024 8506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:00.472003 8505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:00.472486 8500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:54:00.472738 8500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:54:00.473851 8500 hybrid_clock.cc:648] HybridClock initialized: now 1790679240473828 us; error 48 us; skew 500 ppm
I20260929 10:54:00.475111 8500 webserver.cc:460] Webserver started at http://127.20.80.195:39779/ using document root <none> and password file <none>
I20260929 10:54:00.475324 8500 fs_manager.cc:359] Metadata directory not provided
I20260929 10:54:00.475370 8500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:54:00.475464 8500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:54:00.476468 8500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-2/data/instance:
uuid: "8614ab5401a9447291096fe3b116bab5"
format_stamp: "Formatted at 2026-09-29 10:54:00 on dist-test-slave-vqch"
server_key: "ce02f2cfde3d314a23ad44c9cec6cb9f"
server_key_iv: "d94db917505a1361280978105080c05e"
server_key_version: "encryptionkey@0"
I20260929 10:54:00.476756 8500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance:
uuid: "8614ab5401a9447291096fe3b116bab5"
format_stamp: "Formatted at 2026-09-29 10:54:00 on dist-test-slave-vqch"
server_key: "ce02f2cfde3d314a23ad44c9cec6cb9f"
server_key_iv: "d94db917505a1361280978105080c05e"
server_key_version: "encryptionkey@0"
I20260929 10:54:00.478071 8500 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:54:00.478770 8514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:00.478932 8500 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:54:00.479034 8500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-2/wal
uuid: "8614ab5401a9447291096fe3b116bab5"
format_stamp: "Formatted at 2026-09-29 10:54:00 on dist-test-slave-vqch"
server_key: "ce02f2cfde3d314a23ad44c9cec6cb9f"
server_key_iv: "d94db917505a1361280978105080c05e"
server_key_version: "encryptionkey@0"
I20260929 10:54:00.479110 8500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:54:00.504025 8500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:54:00.504333 8500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:54:00.504487 8500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:54:00.504719 8500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:54:00.505093 8500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:54:00.505146 8500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:00.505199 8500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:54:00.505225 8500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:00.510998 8500 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.195:34479
I20260929 10:54:00.511049 8630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.195:34479 every 8 connection(s)
I20260929 10:54:00.511361 8500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-2/data/info.pb
I20260929 10:54:00.514467 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 8500
I20260929 10:54:00.514545 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-2/wal/instance
I20260929 10:54:00.514734 20803 external_mini_cluster.cc:1428] Setting key e428d8e5f4171b6009876ee3e4ece1b5
I20260929 10:54:00.516906 8216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:00.512117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:36153 (local address 127.20.80.254:43629)
0929 10:54:00.512161 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:00.512163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:00.512403 (+ 240us) server_negotiation.cc:407] Connection header received
0929 10:54:00.512877 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:00.512878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:00.512888 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:00.512948 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:00.513631 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.514113 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.515321 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.515600 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.515665 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:00.516618 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:00.516619 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:00.516620 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:54:00.516624 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:00.516639 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:54:00.516642 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:00.516717 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:00.516817 (+ 100us) server_negotiation.cc:299] Negotiation successful
0929 10:54:00.516840 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 10:54:00.517158 8633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:00.512084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:43629 (local address 127.20.80.195:36153)
0929 10:54:00.512265 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:00.512305 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 10:54:00.512797 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:54:00.513063 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:54:00.513067 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:54:00.513082 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:54:00.513561 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:00.513565 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.514206 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.514207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:00.515242 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:00.515245 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.515686 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.515687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:00.515851 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:00.516466 (+ 615us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:54:00.516467 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:54:00.516493 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:54:00.516514 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:54:00.516710 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:54:00.516715 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:54:00.516718 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:54:00.516806 (+ 88us) client_negotiation.cc:240] Negotiation successful
0929 10:54:00.516897 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
I20260929 10:54:00.517639 8631 heartbeater.cc:347] Connected to a master server at 127.20.80.254:43629
I20260929 10:54:00.517771 8631 heartbeater.cc:464] Registering TS with master...
I20260929 10:54:00.517967 8631 heartbeater.cc:511] Master 127.20.80.254:43629 requested a full tablet report, sending...
I20260929 10:54:00.518302 8177 ts_manager.cc:194] Registered new tserver with Master: 8614ab5401a9447291096fe3b116bab5 (127.20.80.195:34479)
I20260929 10:54:00.518667 8177 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.80.195:36153
I20260929 10:54:00.526535 20803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:54:00.530889 8216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:00.527909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56020 (local address 127.20.80.254:43629)
0929 10:54:00.527984 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:00.527987 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:00.528088 (+ 101us) server_negotiation.cc:407] Connection header received
0929 10:54:00.528166 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:00.528167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:00.528177 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:00.528231 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:00.528411 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.528900 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.529672 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.529875 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.529958 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:00.530657 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:00.530659 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:00.530659 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:54:00.530663 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:00.530677 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:54:00.530681 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:00.530728 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:00.530784 (+ 56us) server_negotiation.cc:299] Negotiation successful
0929 10:54:00.530807 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 10:54:00.532629 8177 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56020:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:54:00.534188 8177 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:54:00.540817 8644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:00.536676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55596 (local address 127.20.80.193:42181)
0929 10:54:00.536859 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:00.536863 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:00.537017 (+ 154us) server_negotiation.cc:407] Connection header received
0929 10:54:00.537101 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:00.537104 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:00.537120 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:00.537262 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:54:00.537479 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.538254 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.539919 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.540412 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.540504 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:00.540602 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:00.540686 (+ 84us) server_negotiation.cc:299] Negotiation successful
0929 10:54:00.540726 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":78,"threads_started":1}
I20260929 10:54:00.540851 8641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:00.536387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:42181 (local address 127.0.0.1:55596)
0929 10:54:00.536916 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:00.536927 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:54:00.537024 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:54:00.537338 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:54:00.537339 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:54:00.537342 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:54:00.537395 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:00.537397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.538348 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.538349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:00.539817 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:00.539820 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.540506 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.540507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:00.540582 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:00.540594 (+ 12us) client_negotiation.cc:769] Sending connection context
0929 10:54:00.540670 (+ 76us) client_negotiation.cc:240] Negotiation successful
0929 10:54:00.540725 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"mutex_wait_us":131,"thread_start_us":59,"threads_started":1}
I20260929 10:54:00.540864 8639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:00.536387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.195:34479 (local address 127.0.0.1:42306)
0929 10:54:00.536556 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:00.536595 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 10:54:00.536660 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:54:00.536933 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:54:00.536936 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:54:00.536940 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:54:00.537045 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:00.537047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.538393 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.538394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:00.539708 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:00.539711 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.540616 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.540617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:00.540677 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:00.540681 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:54:00.540712 (+ 31us) client_negotiation.cc:240] Negotiation successful
0929 10:54:00.540756 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":118,"spinlock_wait_cycles":768,"thread_start_us":104,"threads_started":1}
I20260929 10:54:00.541440 8643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:00.536631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42548 (local address 127.20.80.194:34943)
0929 10:54:00.536865 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:00.536869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:00.537235 (+ 366us) server_negotiation.cc:407] Connection header received
0929 10:54:00.537283 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:00.537285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:00.537300 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:00.537370 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:54:00.537463 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.538498 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.539900 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.540560 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.540622 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:00.540725 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:00.541318 (+ 593us) server_negotiation.cc:299] Negotiation successful
0929 10:54:00.541363 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":122,"threads_started":1}
I20260929 10:54:00.541561 8642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:00.536560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42306 (local address 127.20.80.195:34479)
0929 10:54:00.536745 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:00.536749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:00.536759 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:54:00.536800 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:00.536802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:00.536819 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:00.536904 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:54:00.537138 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.538278 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.539795 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.540507 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.541228 (+ 721us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:00.541342 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:00.541365 (+ 23us) server_negotiation.cc:299] Negotiation successful
0929 10:54:00.541400 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":79,"threads_started":1}
I20260929 10:54:00.541649 8640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:00.536397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:34943 (local address 127.0.0.1:42548)
0929 10:54:00.537117 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:00.537126 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:54:00.537180 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:54:00.537381 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:54:00.537382 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:54:00.537383 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:54:00.537413 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:00.537414 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.538544 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.538545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:00.539821 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:00.539823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.540757 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.540758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:00.540812 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:00.540815 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:54:00.540862 (+ 47us) client_negotiation.cc:240] Negotiation successful
0929 10:54:00.540904 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"mutex_wait_us":85,"thread_start_us":75,"threads_started":1}
I20260929 10:54:00.541692 8300 tablet_service.cc:1467] Processing CreateTablet for tablet 7719b6c619bd410dbc388c1b58744f9c (DEFAULT_TABLE table=test-table [id=bb392e4f18c449f285cffb4bcb68a051]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:54:00.542132 8300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7719b6c619bd410dbc388c1b58744f9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:54:00.542286 8431 tablet_service.cc:1467] Processing CreateTablet for tablet 7719b6c619bd410dbc388c1b58744f9c (DEFAULT_TABLE table=test-table [id=bb392e4f18c449f285cffb4bcb68a051]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:54:00.542619 8431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7719b6c619bd410dbc388c1b58744f9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:54:00.545102 8646 tablet_bootstrap.cc:492] T 7719b6c619bd410dbc388c1b58744f9c P 3394dcdad00f42f485541aa317c6a5f2: Bootstrap starting.
I20260929 10:54:00.545105 8645 tablet_bootstrap.cc:492] T 7719b6c619bd410dbc388c1b58744f9c P 282cd0e219024df2962adae8c0ead4b3: Bootstrap starting.
I20260929 10:54:00.545459 8564 tablet_service.cc:1467] Processing CreateTablet for tablet 7719b6c619bd410dbc388c1b58744f9c (DEFAULT_TABLE table=test-table [id=bb392e4f18c449f285cffb4bcb68a051]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:54:00.545763 8646 tablet_bootstrap.cc:654] T 7719b6c619bd410dbc388c1b58744f9c P 3394dcdad00f42f485541aa317c6a5f2: Neither blocks nor log segments found. Creating new log.
I20260929 10:54:00.545809 8564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7719b6c619bd410dbc388c1b58744f9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:54:00.545921 8645 tablet_bootstrap.cc:654] T 7719b6c619bd410dbc388c1b58744f9c P 282cd0e219024df2962adae8c0ead4b3: Neither blocks nor log segments found. Creating new log.
I20260929 10:54:00.546094 8646 log.cc:824] T 7719b6c619bd410dbc388c1b58744f9c P 3394dcdad00f42f485541aa317c6a5f2: Log is configured to *not* fsync() on all Append() calls
I20260929 10:54:00.546233 8645 log.cc:824] T 7719b6c619bd410dbc388c1b58744f9c P 282cd0e219024df2962adae8c0ead4b3: Log is configured to *not* fsync() on all Append() calls
I20260929 10:54:00.546820 8646 tablet_bootstrap.cc:492] T 7719b6c619bd410dbc388c1b58744f9c P 3394dcdad00f42f485541aa317c6a5f2: No bootstrap required, opened a new log
I20260929 10:54:00.546903 8646 ts_tablet_manager.cc:1397] T 7719b6c619bd410dbc388c1b58744f9c P 3394dcdad00f42f485541aa317c6a5f2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:54:00.548338 8649 tablet_bootstrap.cc:492] T 7719b6c619bd410dbc388c1b58744f9c P 8614ab5401a9447291096fe3b116bab5: Bootstrap starting.
I20260929 10:54:00.548910 8649 tablet_bootstrap.cc:654] T 7719b6c619bd410dbc388c1b58744f9c P 8614ab5401a9447291096fe3b116bab5: Neither blocks nor log segments found. Creating new log.
I20260929 10:54:00.548825 8646 raft_consensus.cc:348] T 7719b6c619bd410dbc388c1b58744f9c P 3394dcdad00f42f485541aa317c6a5f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8614ab5401a9447291096fe3b116bab5" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 34479 } } peers { permanent_uuid: "282cd0e219024df2962adae8c0ead4b3" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 42181 } } peers { permanent_uuid: "3394dcdad00f42f485541aa317c6a5f2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34943 } }
I20260929 10:54:00.548967 8646 raft_consensus.cc:374] T 7719b6c619bd410dbc388c1b58744f9c P 3394dcdad00f42f485541aa317c6a5f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:54:00.549031 8646 raft_consensus.cc:729] T 7719b6c619bd410dbc388c1b58744f9c P 3394dcdad00f42f485541aa317c6a5f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3394dcdad00f42f485541aa317c6a5f2, State: Initialized, Role: FOLLOWER
I20260929 10:54:00.549146 8646 consensus_queue.cc:260] T 7719b6c619bd410dbc388c1b58744f9c P 3394dcdad00f42f485541aa317c6a5f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8614ab5401a9447291096fe3b116bab5" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 34479 } } peers { permanent_uuid: "282cd0e219024df2962adae8c0ead4b3" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 42181 } } peers { permanent_uuid: "3394dcdad00f42f485541aa317c6a5f2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34943 } }
I20260929 10:54:00.549242 8649 log.cc:824] T 7719b6c619bd410dbc388c1b58744f9c P 8614ab5401a9447291096fe3b116bab5: Log is configured to *not* fsync() on all Append() calls
I20260929 10:54:00.549412 8646 ts_tablet_manager.cc:1428] T 7719b6c619bd410dbc388c1b58744f9c P 3394dcdad00f42f485541aa317c6a5f2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 10:54:00.549480 8497 heartbeater.cc:503] Master 127.20.80.254:43629 was elected leader, sending a full tablet report...
I20260929 10:54:00.549929 8645 tablet_bootstrap.cc:492] T 7719b6c619bd410dbc388c1b58744f9c P 282cd0e219024df2962adae8c0ead4b3: No bootstrap required, opened a new log
I20260929 10:54:00.549981 8649 tablet_bootstrap.cc:492] T 7719b6c619bd410dbc388c1b58744f9c P 8614ab5401a9447291096fe3b116bab5: No bootstrap required, opened a new log
I20260929 10:54:00.550034 8645 ts_tablet_manager.cc:1397] T 7719b6c619bd410dbc388c1b58744f9c P 282cd0e219024df2962adae8c0ead4b3: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260929 10:54:00.550068 8649 ts_tablet_manager.cc:1397] T 7719b6c619bd410dbc388c1b58744f9c P 8614ab5401a9447291096fe3b116bab5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:54:00.552196 8645 raft_consensus.cc:348] T 7719b6c619bd410dbc388c1b58744f9c P 282cd0e219024df2962adae8c0ead4b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8614ab5401a9447291096fe3b116bab5" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 34479 } } peers { permanent_uuid: "282cd0e219024df2962adae8c0ead4b3" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 42181 } } peers { permanent_uuid: "3394dcdad00f42f485541aa317c6a5f2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34943 } }
I20260929 10:54:00.552305 8649 raft_consensus.cc:348] T 7719b6c619bd410dbc388c1b58744f9c P 8614ab5401a9447291096fe3b116bab5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8614ab5401a9447291096fe3b116bab5" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 34479 } } peers { permanent_uuid: "282cd0e219024df2962adae8c0ead4b3" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 42181 } } peers { permanent_uuid: "3394dcdad00f42f485541aa317c6a5f2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34943 } }
I20260929 10:54:00.552402 8645 raft_consensus.cc:374] T 7719b6c619bd410dbc388c1b58744f9c P 282cd0e219024df2962adae8c0ead4b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:54:00.552424 8649 raft_consensus.cc:374] T 7719b6c619bd410dbc388c1b58744f9c P 8614ab5401a9447291096fe3b116bab5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:54:00.552456 8645 raft_consensus.cc:729] T 7719b6c619bd410dbc388c1b58744f9c P 282cd0e219024df2962adae8c0ead4b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 282cd0e219024df2962adae8c0ead4b3, State: Initialized, Role: FOLLOWER
I20260929 10:54:00.552486 8649 raft_consensus.cc:729] T 7719b6c619bd410dbc388c1b58744f9c P 8614ab5401a9447291096fe3b116bab5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8614ab5401a9447291096fe3b116bab5, State: Initialized, Role: FOLLOWER
I20260929 10:54:00.552594 8645 consensus_queue.cc:260] T 7719b6c619bd410dbc388c1b58744f9c P 282cd0e219024df2962adae8c0ead4b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8614ab5401a9447291096fe3b116bab5" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 34479 } } peers { permanent_uuid: "282cd0e219024df2962adae8c0ead4b3" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 42181 } } peers { permanent_uuid: "3394dcdad00f42f485541aa317c6a5f2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34943 } }
I20260929 10:54:00.552598 8649 consensus_queue.cc:260] T 7719b6c619bd410dbc388c1b58744f9c P 8614ab5401a9447291096fe3b116bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8614ab5401a9447291096fe3b116bab5" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 34479 } } peers { permanent_uuid: "282cd0e219024df2962adae8c0ead4b3" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 42181 } } peers { permanent_uuid: "3394dcdad00f42f485541aa317c6a5f2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34943 } }
I20260929 10:54:00.552870 8649 ts_tablet_manager.cc:1428] T 7719b6c619bd410dbc388c1b58744f9c P 8614ab5401a9447291096fe3b116bab5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 10:54:00.552866 8645 ts_tablet_manager.cc:1428] T 7719b6c619bd410dbc388c1b58744f9c P 282cd0e219024df2962adae8c0ead4b3: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260929 10:54:00.552897 8631 heartbeater.cc:503] Master 127.20.80.254:43629 was elected leader, sending a full tablet report...
I20260929 10:54:00.553212 8366 heartbeater.cc:503] Master 127.20.80.254:43629 was elected leader, sending a full tablet report...
I20260929 10:54:00.572938 8653 raft_consensus.cc:482] T 7719b6c619bd410dbc388c1b58744f9c P 8614ab5401a9447291096fe3b116bab5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:54:00.573127 8653 raft_consensus.cc:504] T 7719b6c619bd410dbc388c1b58744f9c P 8614ab5401a9447291096fe3b116bab5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8614ab5401a9447291096fe3b116bab5" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 34479 } } peers { permanent_uuid: "282cd0e219024df2962adae8c0ead4b3" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 42181 } } peers { permanent_uuid: "3394dcdad00f42f485541aa317c6a5f2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34943 } }
I20260929 10:54:00.573490 8653 leader_election.cc:288] T 7719b6c619bd410dbc388c1b58744f9c P 8614ab5401a9447291096fe3b116bab5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 282cd0e219024df2962adae8c0ead4b3 (127.20.80.193:42181), 3394dcdad00f42f485541aa317c6a5f2 (127.20.80.194:34943)
I20260929 10:54:00.576349 8643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:00.573744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:41003 (local address 127.20.80.194:34943)
0929 10:54:00.573793 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:00.573796 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:00.573955 (+ 159us) server_negotiation.cc:407] Connection header received
0929 10:54:00.573985 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:00.573986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:00.573998 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:00.574059 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:54:00.574220 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.574704 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.575649 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.575999 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.576054 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:00.576059 (+ 5us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:00.576237 (+ 178us) server_negotiation.cc:299] Negotiation successful
0929 10:54:00.576282 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260929 10:54:00.576411 8644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:00.573736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.80.195:35297 (local address 127.20.80.193:42181)
0929 10:54:00.573803 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:00.573805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:00.573813 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:54:00.573841 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:00.573842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:00.573853 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:00.573923 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:54:00.574106 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.574625 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.575657 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.576019 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.576107 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:00.576115 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:00.576294 (+ 179us) server_negotiation.cc:299] Negotiation successful
0929 10:54:00.576334 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 10:54:00.576432 8633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:00.573660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:42181 (local address 127.20.80.195:35297)
0929 10:54:00.573715 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:00.573727 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:54:00.573770 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:54:00.573955 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:54:00.573956 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:54:00.573958 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:54:00.574029 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:00.574031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.574707 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.574708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:00.575552 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:00.575555 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.576123 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.576124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:00.576215 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:00.576219 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:54:00.576276 (+ 57us) client_negotiation.cc:240] Negotiation successful
0929 10:54:00.576329 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":39}
I20260929 10:54:00.576357 8654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:00.573662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.194:34943 (local address 127.20.80.195:41003)
0929 10:54:00.573874 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:00.573885 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:54:00.573942 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:54:00.574095 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:54:00.574096 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:54:00.574098 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:54:00.574163 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:00.574165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.574793 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.574793 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:00.575554 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:00.575556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.576084 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.576085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:00.576155 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:00.576158 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:54:00.576210 (+ 52us) client_negotiation.cc:240] Negotiation successful
0929 10:54:00.576238 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":14,"thread_start_us":124,"threads_started":1}
I20260929 10:54:00.576787 8320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7719b6c619bd410dbc388c1b58744f9c" candidate_uuid: "8614ab5401a9447291096fe3b116bab5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "282cd0e219024df2962adae8c0ead4b3" is_pre_election: true
I20260929 10:54:00.576781 8451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7719b6c619bd410dbc388c1b58744f9c" candidate_uuid: "8614ab5401a9447291096fe3b116bab5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3394dcdad00f42f485541aa317c6a5f2" is_pre_election: true
I20260929 10:54:00.576929 8451 raft_consensus.cc:2454] T 7719b6c619bd410dbc388c1b58744f9c P 3394dcdad00f42f485541aa317c6a5f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8614ab5401a9447291096fe3b116bab5 in term 0.
I20260929 10:54:00.576929 8320 raft_consensus.cc:2454] T 7719b6c619bd410dbc388c1b58744f9c P 282cd0e219024df2962adae8c0ead4b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8614ab5401a9447291096fe3b116bab5 in term 0.
I20260929 10:54:00.577171 8518 leader_election.cc:302] T 7719b6c619bd410dbc388c1b58744f9c P 8614ab5401a9447291096fe3b116bab5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3394dcdad00f42f485541aa317c6a5f2, 8614ab5401a9447291096fe3b116bab5; no voters:
I20260929 10:54:00.577291 8653 raft_consensus.cc:2781] T 7719b6c619bd410dbc388c1b58744f9c P 8614ab5401a9447291096fe3b116bab5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:54:00.577343 8653 raft_consensus.cc:482] T 7719b6c619bd410dbc388c1b58744f9c P 8614ab5401a9447291096fe3b116bab5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:54:00.577366 8653 raft_consensus.cc:3037] T 7719b6c619bd410dbc388c1b58744f9c P 8614ab5401a9447291096fe3b116bab5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:54:00.578015 8653 raft_consensus.cc:504] T 7719b6c619bd410dbc388c1b58744f9c P 8614ab5401a9447291096fe3b116bab5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8614ab5401a9447291096fe3b116bab5" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 34479 } } peers { permanent_uuid: "282cd0e219024df2962adae8c0ead4b3" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 42181 } } peers { permanent_uuid: "3394dcdad00f42f485541aa317c6a5f2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34943 } }
I20260929 10:54:00.578158 8653 leader_election.cc:288] T 7719b6c619bd410dbc388c1b58744f9c P 8614ab5401a9447291096fe3b116bab5 [CANDIDATE]: Term 1 election: Requested vote from peers 282cd0e219024df2962adae8c0ead4b3 (127.20.80.193:42181), 3394dcdad00f42f485541aa317c6a5f2 (127.20.80.194:34943)
I20260929 10:54:00.578311 8451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7719b6c619bd410dbc388c1b58744f9c" candidate_uuid: "8614ab5401a9447291096fe3b116bab5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3394dcdad00f42f485541aa317c6a5f2"
I20260929 10:54:00.578387 8451 raft_consensus.cc:3037] T 7719b6c619bd410dbc388c1b58744f9c P 3394dcdad00f42f485541aa317c6a5f2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:54:00.579013 8451 raft_consensus.cc:2454] T 7719b6c619bd410dbc388c1b58744f9c P 3394dcdad00f42f485541aa317c6a5f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8614ab5401a9447291096fe3b116bab5 in term 1.
I20260929 10:54:00.579159 8518 leader_election.cc:302] T 7719b6c619bd410dbc388c1b58744f9c P 8614ab5401a9447291096fe3b116bab5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3394dcdad00f42f485541aa317c6a5f2, 8614ab5401a9447291096fe3b116bab5; no voters:
I20260929 10:54:00.579149 8320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7719b6c619bd410dbc388c1b58744f9c" candidate_uuid: "8614ab5401a9447291096fe3b116bab5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "282cd0e219024df2962adae8c0ead4b3"
I20260929 10:54:00.579223 8320 raft_consensus.cc:3037] T 7719b6c619bd410dbc388c1b58744f9c P 282cd0e219024df2962adae8c0ead4b3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:54:00.579272 8653 raft_consensus.cc:2781] T 7719b6c619bd410dbc388c1b58744f9c P 8614ab5401a9447291096fe3b116bab5 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:54:00.579445 8653 raft_consensus.cc:686] T 7719b6c619bd410dbc388c1b58744f9c P 8614ab5401a9447291096fe3b116bab5 [term 1 LEADER]: Becoming Leader. State: Replica: 8614ab5401a9447291096fe3b116bab5, State: Running, Role: LEADER
I20260929 10:54:00.579545 8653 consensus_queue.cc:237] T 7719b6c619bd410dbc388c1b58744f9c P 8614ab5401a9447291096fe3b116bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8614ab5401a9447291096fe3b116bab5" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 34479 } } peers { permanent_uuid: "282cd0e219024df2962adae8c0ead4b3" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 42181 } } peers { permanent_uuid: "3394dcdad00f42f485541aa317c6a5f2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34943 } }
I20260929 10:54:00.580139 8320 raft_consensus.cc:2454] T 7719b6c619bd410dbc388c1b58744f9c P 282cd0e219024df2962adae8c0ead4b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8614ab5401a9447291096fe3b116bab5 in term 1.
I20260929 10:54:00.580325 8176 catalog_manager.cc:5495] T 7719b6c619bd410dbc388c1b58744f9c P 8614ab5401a9447291096fe3b116bab5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8614ab5401a9447291096fe3b116bab5 (127.20.80.195). New cstate: current_term: 1 leader_uuid: "8614ab5401a9447291096fe3b116bab5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8614ab5401a9447291096fe3b116bab5" member_type: VOTER last_known_addr { host: "127.20.80.195" port: 34479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "282cd0e219024df2962adae8c0ead4b3" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 42181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3394dcdad00f42f485541aa317c6a5f2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34943 } health_report { overall_health: UNKNOWN } } }
I20260929 10:54:00.594507 8642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:00.592092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42322 (local address 127.20.80.195:34479)
0929 10:54:00.592139 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:00.592141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:00.592150 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:54:00.592217 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:00.592218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:00.592236 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:00.592321 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:54:00.592451 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.592929 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.593711 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:00.593915 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:00.593999 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:00.594165 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:54:00.594266 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:54:00.594358 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:00.594388 (+ 30us) server_negotiation.cc:299] Negotiation successful
0929 10:54:00.594403 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 10:54:00.596256 8320 raft_consensus.cc:1261] T 7719b6c619bd410dbc388c1b58744f9c P 282cd0e219024df2962adae8c0ead4b3 [term 1 FOLLOWER]: Refusing update from remote peer 8614ab5401a9447291096fe3b116bab5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:54:00.596257 8451 raft_consensus.cc:1261] T 7719b6c619bd410dbc388c1b58744f9c P 3394dcdad00f42f485541aa317c6a5f2 [term 1 FOLLOWER]: Refusing update from remote peer 8614ab5401a9447291096fe3b116bab5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:54:00.596606 8655 consensus_queue.cc:1035] T 7719b6c619bd410dbc388c1b58744f9c P 8614ab5401a9447291096fe3b116bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "282cd0e219024df2962adae8c0ead4b3" member_type: VOTER last_known_addr { host: "127.20.80.193" port: 42181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:54:00.596755 8653 consensus_queue.cc:1035] T 7719b6c619bd410dbc388c1b58744f9c P 8614ab5401a9447291096fe3b116bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3394dcdad00f42f485541aa317c6a5f2" member_type: VOTER last_known_addr { host: "127.20.80.194" port: 34943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:54:00.599351 8658 mvcc.cc:205] Tried to move back new op lower bound from 7334622169479462912 to 7334622169414397952. Current Snapshot: MvccSnapshot[applied={T|T < 7334622169479462912}]
I20260929 10:54:00.599647 8659 mvcc.cc:205] Tried to move back new op lower bound from 7334622169479462912 to 7334622169414397952. Current Snapshot: MvccSnapshot[applied={T|T < 7334622169479462912}]
I20260929 10:54:00.601639 8175 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:54:00.601758 8175 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:54:00.602527 8175 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2864e06b3c6f4978ba659d46923fbe00: Sending DeleteTablet for 3 replicas of tablet 7719b6c619bd410dbc388c1b58744f9c
I20260929 10:54:00.602913 8431 tablet_service.cc:1514] Processing DeleteTablet for tablet 7719b6c619bd410dbc388c1b58744f9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:54:00 UTC) from {username='slave'} at 127.0.0.1:42548
I20260929 10:54:00.602932 8300 tablet_service.cc:1514] Processing DeleteTablet for tablet 7719b6c619bd410dbc388c1b58744f9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:54:00 UTC) from {username='slave'} at 127.0.0.1:55596
I20260929 10:54:00.603149 8564 tablet_service.cc:1514] Processing DeleteTablet for tablet 7719b6c619bd410dbc388c1b58744f9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:54:00 UTC) from {username='slave'} at 127.0.0.1:42306
I20260929 10:54:00.603180 8668 tablet_replica.cc:330] T 7719b6c619bd410dbc388c1b58744f9c P 3394dcdad00f42f485541aa317c6a5f2: stopping tablet replica
I20260929 10:54:00.603335 8668 raft_consensus.cc:2229] T 7719b6c619bd410dbc388c1b58744f9c P 3394dcdad00f42f485541aa317c6a5f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:54:00.603436 8670 tablet_replica.cc:330] T 7719b6c619bd410dbc388c1b58744f9c P 8614ab5401a9447291096fe3b116bab5: stopping tablet replica
I20260929 10:54:00.603523 8670 raft_consensus.cc:2229] T 7719b6c619bd410dbc388c1b58744f9c P 8614ab5401a9447291096fe3b116bab5 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:54:00.603669 8668 raft_consensus.cc:2258] T 7719b6c619bd410dbc388c1b58744f9c P 3394dcdad00f42f485541aa317c6a5f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:54:00.603693 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 8237
I20260929 10:54:00.603641 8670 raft_consensus.cc:2258] T 7719b6c619bd410dbc388c1b58744f9c P 8614ab5401a9447291096fe3b116bab5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:54:00.604794 8668 ts_tablet_manager.cc:1905] T 7719b6c619bd410dbc388c1b58744f9c P 3394dcdad00f42f485541aa317c6a5f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:54:00.606477 8668 ts_tablet_manager.cc:1918] T 7719b6c619bd410dbc388c1b58744f9c P 3394dcdad00f42f485541aa317c6a5f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:54:00.606565 8668 log.cc:1196] T 7719b6c619bd410dbc388c1b58744f9c P 3394dcdad00f42f485541aa317c6a5f2: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-1/wal/wals/7719b6c619bd410dbc388c1b58744f9c
I20260929 10:54:00.606802 8668 ts_tablet_manager.cc:1939] T 7719b6c619bd410dbc388c1b58744f9c P 3394dcdad00f42f485541aa317c6a5f2: Deleting consensus metadata
I20260929 10:54:00.607194 8164 catalog_manager.cc:4841] TS 3394dcdad00f42f485541aa317c6a5f2 (127.20.80.194:34943): tablet 7719b6c619bd410dbc388c1b58744f9c (table test-table [id=bb392e4f18c449f285cffb4bcb68a051]) successfully deleted
I20260929 10:54:00.609339 8670 ts_tablet_manager.cc:1905] T 7719b6c619bd410dbc388c1b58744f9c P 8614ab5401a9447291096fe3b116bab5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260929 10:54:00.609673 8162 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.80.193:42181 (error 108)
I20260929 10:54:00.610194 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 8369
I20260929 10:54:00.610844 8670 ts_tablet_manager.cc:1918] T 7719b6c619bd410dbc388c1b58744f9c P 8614ab5401a9447291096fe3b116bab5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:54:00.610915 8670 log.cc:1196] T 7719b6c619bd410dbc388c1b58744f9c P 8614ab5401a9447291096fe3b116bab5: Deleting WAL directory at /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790678996126930-20803-0/minicluster-data/ts-2/wal/wals/7719b6c619bd410dbc388c1b58744f9c
I20260929 10:54:00.611147 8670 ts_tablet_manager.cc:1939] T 7719b6c619bd410dbc388c1b58744f9c P 8614ab5401a9447291096fe3b116bab5: Deleting consensus metadata
I20260929 10:54:00.611637 8163 catalog_manager.cc:4841] TS 8614ab5401a9447291096fe3b116bab5 (127.20.80.195:34479): tablet 7719b6c619bd410dbc388c1b58744f9c (table test-table [id=bb392e4f18c449f285cffb4bcb68a051]) successfully deleted
I20260929 10:54:00.613385 8640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:00.613248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.193:42181 (local address 127.0.0.1:55610)
0929 10:54:00.613307 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:00.613349 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.193:42181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260929 10:54:00.613564 8162 catalog_manager.cc:4584] TS 282cd0e219024df2962adae8c0ead4b3 (127.20.80.193:42181): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7719b6c619bd410dbc388c1b58744f9c: Network error: Client connection negotiation failed: client connection to 127.20.80.193:42181: connect: Connection refused (error 111)
I20260929 10:54:00.617273 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 8500
I20260929 10:54:00.622005 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 8146
2026-09-29T10:54:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (738 ms)
[----------] 6 tests from EncryptionPolicyTest (9055 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:54:00 dist-test-slave-vqch krb5kdc[8674](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:54:00 dist-test-slave-vqch krb5kdc[8674](info): set up 2 sockets
Sep 29 10:54:00 dist-test-slave-vqch krb5kdc[8674](info): commencing operation
krb5kdc: starting...
W20260929 10:54:01.664765 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:54:01 dist-test-slave-vqch krb5kdc[8674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679241, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:54:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:54:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 10:54:01.784359 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34905
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34905
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:54:01.866189 8687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:54:01.866434 8687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:54:01.866472 8687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:54:01.867606 8687 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:54:01.867682 8687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:54:01.867707 8687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:54:01.867736 8687 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:54:01.867767 8687 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:54:01.869288 8687 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34905
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:54:01.869566 8687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:54:01.869799 8687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:54:01.874532 8692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:01.874532 8695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:01.874593 8693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:01.874725 8687 server_base.cc:1034] running on GCE node
I20260929 10:54:01.875082 8687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:54:01.875378 8687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:54:01.876525 8687 hybrid_clock.cc:648] HybridClock initialized: now 1790679241876502 us; error 35 us; skew 500 ppm
Sep 29 10:54:01 dist-test-slave-vqch krb5kdc[8674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679241, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:54:01.884193 8687 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 10:54:01.884682 8687 webserver.cc:460] Webserver started at http://127.0.0.1:38647/ using document root <none> and password file <none>
I20260929 10:54:01.884941 8687 fs_manager.cc:359] Metadata directory not provided
I20260929 10:54:01.885031 8687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:54:01.885186 8687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:54:01.886314 8687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "aaa638be74ad453ba46e2fb1a6d814e6"
format_stamp: "Formatted at 2026-09-29 10:54:01 on dist-test-slave-vqch"
server_key: "572a7fb16eb7a28b01ae7a2243cd698a"
server_key_iv: "07df0626253fffea34732bfac6fd7ac9"
server_key_version: "encryptionkey@0"
I20260929 10:54:01.886682 8687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "aaa638be74ad453ba46e2fb1a6d814e6"
format_stamp: "Formatted at 2026-09-29 10:54:01 on dist-test-slave-vqch"
server_key: "572a7fb16eb7a28b01ae7a2243cd698a"
server_key_iv: "07df0626253fffea34732bfac6fd7ac9"
server_key_version: "encryptionkey@0"
I20260929 10:54:01.888036 8687 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 10:54:01.888777 8702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:01.888944 8687 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:54:01.889060 8687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "aaa638be74ad453ba46e2fb1a6d814e6"
format_stamp: "Formatted at 2026-09-29 10:54:01 on dist-test-slave-vqch"
server_key: "572a7fb16eb7a28b01ae7a2243cd698a"
server_key_iv: "07df0626253fffea34732bfac6fd7ac9"
server_key_version: "encryptionkey@0"
I20260929 10:54:01.889142 8687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:54:01.900193 8687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:54:01.902506 8687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:54:01.902673 8687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:54:01.906781 8687 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34905
I20260929 10:54:01.907166 8687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:54:01.907155 8755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34905 every 8 connection(s)
I20260929 10:54:01.907943 8756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:54:01.909121 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 8687
I20260929 10:54:01.909250 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:54:01.909425 20803 external_mini_cluster.cc:1428] Setting key 7d00559b449d88a12b84500869e743a0
I20260929 10:54:01.910925 8756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aaa638be74ad453ba46e2fb1a6d814e6: Bootstrap starting.
I20260929 10:54:01.911711 8756 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aaa638be74ad453ba46e2fb1a6d814e6: Neither blocks nor log segments found. Creating new log.
I20260929 10:54:01.912045 8756 log.cc:824] T 00000000000000000000000000000000 P aaa638be74ad453ba46e2fb1a6d814e6: Log is configured to *not* fsync() on all Append() calls
I20260929 10:54:01.912822 8756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aaa638be74ad453ba46e2fb1a6d814e6: No bootstrap required, opened a new log
I20260929 10:54:01.915445 8756 raft_consensus.cc:348] T 00000000000000000000000000000000 P aaa638be74ad453ba46e2fb1a6d814e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaa638be74ad453ba46e2fb1a6d814e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34905 } }
I20260929 10:54:01.915663 8756 raft_consensus.cc:374] T 00000000000000000000000000000000 P aaa638be74ad453ba46e2fb1a6d814e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:54:01.915709 8756 raft_consensus.cc:729] T 00000000000000000000000000000000 P aaa638be74ad453ba46e2fb1a6d814e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaa638be74ad453ba46e2fb1a6d814e6, State: Initialized, Role: FOLLOWER
I20260929 10:54:01.915875 8756 consensus_queue.cc:260] T 00000000000000000000000000000000 P aaa638be74ad453ba46e2fb1a6d814e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaa638be74ad453ba46e2fb1a6d814e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34905 } }
I20260929 10:54:01.915984 8756 raft_consensus.cc:388] T 00000000000000000000000000000000 P aaa638be74ad453ba46e2fb1a6d814e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:54:01.916026 8756 raft_consensus.cc:482] T 00000000000000000000000000000000 P aaa638be74ad453ba46e2fb1a6d814e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:54:01.916079 8756 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aaa638be74ad453ba46e2fb1a6d814e6 [term 0 FOLLOWER]: Advancing to term 1
Sep 29 10:54:01 dist-test-slave-vqch krb5kdc[8674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679241, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 10:54:01.916862 8756 raft_consensus.cc:504] T 00000000000000000000000000000000 P aaa638be74ad453ba46e2fb1a6d814e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaa638be74ad453ba46e2fb1a6d814e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34905 } }
I20260929 10:54:01.917032 8756 leader_election.cc:302] T 00000000000000000000000000000000 P aaa638be74ad453ba46e2fb1a6d814e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aaa638be74ad453ba46e2fb1a6d814e6; no voters:
I20260929 10:54:01.917229 8756 leader_election.cc:288] T 00000000000000000000000000000000 P aaa638be74ad453ba46e2fb1a6d814e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:54:01.917255 8761 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aaa638be74ad453ba46e2fb1a6d814e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:54:01.917376 8761 raft_consensus.cc:686] T 00000000000000000000000000000000 P aaa638be74ad453ba46e2fb1a6d814e6 [term 1 LEADER]: Becoming Leader. State: Replica: aaa638be74ad453ba46e2fb1a6d814e6, State: Running, Role: LEADER
I20260929 10:54:01.917467 8761 consensus_queue.cc:237] T 00000000000000000000000000000000 P aaa638be74ad453ba46e2fb1a6d814e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaa638be74ad453ba46e2fb1a6d814e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34905 } }
I20260929 10:54:01.917508 8756 sys_catalog.cc:564] T 00000000000000000000000000000000 P aaa638be74ad453ba46e2fb1a6d814e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:54:01.917790 8761 sys_catalog.cc:455] T 00000000000000000000000000000000 P aaa638be74ad453ba46e2fb1a6d814e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aaa638be74ad453ba46e2fb1a6d814e6. Latest consensus state: current_term: 1 leader_uuid: "aaa638be74ad453ba46e2fb1a6d814e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaa638be74ad453ba46e2fb1a6d814e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34905 } } }
I20260929 10:54:01.917871 8761 sys_catalog.cc:458] T 00000000000000000000000000000000 P aaa638be74ad453ba46e2fb1a6d814e6 [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:01.917807 8762 sys_catalog.cc:455] T 00000000000000000000000000000000 P aaa638be74ad453ba46e2fb1a6d814e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aaa638be74ad453ba46e2fb1a6d814e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaa638be74ad453ba46e2fb1a6d814e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34905 } } }
I20260929 10:54:01.918063 8762 sys_catalog.cc:458] T 00000000000000000000000000000000 P aaa638be74ad453ba46e2fb1a6d814e6 [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:01.918128 8765 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:54:01.918644 8765 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:54:01.920383 8765 catalog_manager.cc:1347] Generated new cluster ID: 3b0d0afb15794d9ab4bf2d9ecc4fb9dc
I20260929 10:54:01.920437 8765 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:54:01.921518 8759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:01.909993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56989 (local address 127.0.0.1:34905)
0929 10:54:01.910174 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:01.910190 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:01.910216 (+ 26us) server_negotiation.cc:407] Connection header received
0929 10:54:01.910620 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:01.910627 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:01.910683 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:01.910799 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:01.911705 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:01.912711 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:01.913818 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:01.914237 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:01.914313 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:01.917535 (+ 3222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:01.917539 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:01.917540 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:01.917555 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:01.919733 (+ 2178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:01.920182 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:01.920185 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:01.920185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:01.920318 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:01.920585 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:01.920586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:01.920586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:01.920853 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:01.921075 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:01.921176 (+ 101us) server_negotiation.cc:299] Negotiation successful
0929 10:54:01.921374 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":80,"threads_started":1}
I20260929 10:54:01.922391 20803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:54:01.930136 8759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:01.923793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40617 (local address 127.0.0.1:34905)
0929 10:54:01.923832 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:01.923834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:01.923957 (+ 123us) server_negotiation.cc:407] Connection header received
0929 10:54:01.924030 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:01.924030 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:01.924040 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:01.924087 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:01.924614 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:01.925225 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:01.925943 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:01.926177 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:01.926266 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:01.927460 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:01.927461 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:01.927461 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:01.927468 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:01.928919 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:01.929319 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:01.929320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:01.929320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:01.929380 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:01.929604 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:01.929612 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:01.929612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:01.929723 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:01.929835 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:01.929960 (+ 125us) server_negotiation.cc:299] Negotiation successful
0929 10:54:01.930023 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 10:54:01.940017 8765 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:54:01.940747 8765 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:54:01.956367 8765 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aaa638be74ad453ba46e2fb1a6d814e6: Generated new TSK 0
I20260929 10:54:01.956621 8765 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:54:01.960490 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 8687
W20260929 10:54:01.964625 8672 connection.cc:535] client connection to 127.0.0.1:34905 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34905): Connection reset by peer (error 104)
2026-09-29T10:54:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1336 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:54:02 dist-test-slave-vqch krb5kdc[8787](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:54:02 dist-test-slave-vqch krb5kdc[8787](info): set up 2 sockets
Sep 29 10:54:02 dist-test-slave-vqch krb5kdc[8787](info): commencing operation
krb5kdc: starting...
W20260929 10:54:03.004796 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:54:03 dist-test-slave-vqch krb5kdc[8787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679243, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:54:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:54:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 10:54:03.123793 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35193
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35193
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:54:03.204798 8800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:54:03.205044 8800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:54:03.205068 8800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:54:03.206272 8800 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:54:03.206315 8800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:54:03.206329 8800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:54:03.206342 8800 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:54:03.206354 8800 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:54:03.207909 8800 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35193
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:54:03.208182 8800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:54:03.208413 8800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:54:03.213675 8808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:03.213661 8805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:03.213666 8806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:03.213721 8800 server_base.cc:1034] running on GCE node
I20260929 10:54:03.214097 8800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:54:03.214388 8800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:54:03.215577 8800 hybrid_clock.cc:648] HybridClock initialized: now 1790679243215548 us; error 44 us; skew 500 ppm
Sep 29 10:54:03 dist-test-slave-vqch krb5kdc[8787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679243, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:54:03.218286 8800 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 10:54:03.218735 8800 webserver.cc:460] Webserver started at http://127.0.0.1:34027/ using document root <none> and password file <none>
I20260929 10:54:03.218986 8800 fs_manager.cc:359] Metadata directory not provided
I20260929 10:54:03.219033 8800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:54:03.219161 8800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:54:03.220238 8800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "5f9085e2c612465ba27c45ede7ffb173"
format_stamp: "Formatted at 2026-09-29 10:54:03 on dist-test-slave-vqch"
server_key: "ea43724239684606a34a05b72dda5969"
server_key_iv: "1d8a8848b45e8d89d9c83a4cb461ef68"
server_key_version: "encryptionkey@0"
I20260929 10:54:03.220614 8800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "5f9085e2c612465ba27c45ede7ffb173"
format_stamp: "Formatted at 2026-09-29 10:54:03 on dist-test-slave-vqch"
server_key: "ea43724239684606a34a05b72dda5969"
server_key_iv: "1d8a8848b45e8d89d9c83a4cb461ef68"
server_key_version: "encryptionkey@0"
I20260929 10:54:03.222069 8800 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 10:54:03.222918 8815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:03.223145 8800 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 10:54:03.223254 8800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "5f9085e2c612465ba27c45ede7ffb173"
format_stamp: "Formatted at 2026-09-29 10:54:03 on dist-test-slave-vqch"
server_key: "ea43724239684606a34a05b72dda5969"
server_key_iv: "1d8a8848b45e8d89d9c83a4cb461ef68"
server_key_version: "encryptionkey@0"
I20260929 10:54:03.223330 8800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:54:03.234855 8800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:54:03.237289 8800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:54:03.237478 8800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:54:03.242147 8800 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35193
I20260929 10:54:03.242177 8868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35193 every 8 connection(s)
I20260929 10:54:03.242609 8800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:54:03.243494 8869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:54:03.245910 8869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f9085e2c612465ba27c45ede7ffb173: Bootstrap starting.
I20260929 10:54:03.246500 8869 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f9085e2c612465ba27c45ede7ffb173: Neither blocks nor log segments found. Creating new log.
I20260929 10:54:03.246793 8869 log.cc:824] T 00000000000000000000000000000000 P 5f9085e2c612465ba27c45ede7ffb173: Log is configured to *not* fsync() on all Append() calls
I20260929 10:54:03.247491 8869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f9085e2c612465ba27c45ede7ffb173: No bootstrap required, opened a new log
I20260929 10:54:03.248585 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 8800
I20260929 10:54:03.248754 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:54:03.248940 20803 external_mini_cluster.cc:1428] Setting key c069586813426c2c89602f9d07f07343
I20260929 10:54:03.249334 8869 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f9085e2c612465ba27c45ede7ffb173 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f9085e2c612465ba27c45ede7ffb173" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35193 } }
I20260929 10:54:03.249557 8869 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f9085e2c612465ba27c45ede7ffb173 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:54:03.249663 8869 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f9085e2c612465ba27c45ede7ffb173 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f9085e2c612465ba27c45ede7ffb173, State: Initialized, Role: FOLLOWER
I20260929 10:54:03.249830 8869 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f9085e2c612465ba27c45ede7ffb173 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f9085e2c612465ba27c45ede7ffb173" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35193 } }
I20260929 10:54:03.249928 8869 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f9085e2c612465ba27c45ede7ffb173 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:54:03.249969 8869 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f9085e2c612465ba27c45ede7ffb173 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:54:03.250020 8869 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f9085e2c612465ba27c45ede7ffb173 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:54:03.250876 8869 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f9085e2c612465ba27c45ede7ffb173 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f9085e2c612465ba27c45ede7ffb173" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35193 } }
I20260929 10:54:03.250995 8869 leader_election.cc:302] T 00000000000000000000000000000000 P 5f9085e2c612465ba27c45ede7ffb173 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f9085e2c612465ba27c45ede7ffb173; no voters:
I20260929 10:54:03.251173 8869 leader_election.cc:288] T 00000000000000000000000000000000 P 5f9085e2c612465ba27c45ede7ffb173 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:54:03.251222 8874 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f9085e2c612465ba27c45ede7ffb173 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:54:03.251368 8874 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f9085e2c612465ba27c45ede7ffb173 [term 1 LEADER]: Becoming Leader. State: Replica: 5f9085e2c612465ba27c45ede7ffb173, State: Running, Role: LEADER
I20260929 10:54:03.251442 8869 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f9085e2c612465ba27c45ede7ffb173 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:54:03.251499 8874 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f9085e2c612465ba27c45ede7ffb173 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f9085e2c612465ba27c45ede7ffb173" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35193 } }
I20260929 10:54:03.251817 8874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f9085e2c612465ba27c45ede7ffb173 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f9085e2c612465ba27c45ede7ffb173. Latest consensus state: current_term: 1 leader_uuid: "5f9085e2c612465ba27c45ede7ffb173" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f9085e2c612465ba27c45ede7ffb173" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35193 } } }
I20260929 10:54:03.251897 8874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f9085e2c612465ba27c45ede7ffb173 [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:03.253151 8875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f9085e2c612465ba27c45ede7ffb173 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f9085e2c612465ba27c45ede7ffb173" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f9085e2c612465ba27c45ede7ffb173" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35193 } } }
I20260929 10:54:03.253288 8875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f9085e2c612465ba27c45ede7ffb173 [sys.catalog]: This master's current role is: LEADER
W20260929 10:54:03.253414 8890 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5f9085e2c612465ba27c45ede7ffb173: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:54:03.253466 8890 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:54:03.253558 8880 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:54:03.253813 8880 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:54:03.254765 8873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:03.249502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39351 (local address 127.0.0.1:35193)
0929 10:54:03.249679 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:03.249688 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:03.249726 (+ 38us) server_negotiation.cc:407] Connection header received
0929 10:54:03.249855 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:03.249862 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:03.249914 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:03.250049 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:03.250294 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:03.251374 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:03.252571 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:03.253134 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:03.253237 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:03.254005 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:03.254009 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:03.254010 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:54:03.254233 (+ 223us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:03.254271 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:54:03.254285 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:03.254377 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:03.254517 (+ 140us) server_negotiation.cc:299] Negotiation successful
0929 10:54:03.254554 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
I20260929 10:54:03.255371 8880 catalog_manager.cc:1347] Generated new cluster ID: ec1ef262447b478e87fdf58cd4e57b63
I20260929 10:54:03.255468 8880 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:54:03.255789 20803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:54:03.261143 8873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:03.257263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34545 (local address 127.0.0.1:35193)
0929 10:54:03.257301 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:03.257303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:03.257418 (+ 115us) server_negotiation.cc:407] Connection header received
0929 10:54:03.257560 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:03.257562 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:03.257572 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:03.257602 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:03.257809 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:03.258522 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:03.259571 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:03.259897 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:03.259939 (+ 42us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:03.260776 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:03.260778 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:03.260778 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:54:03.260785 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:03.260803 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:54:03.260808 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:03.260845 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:03.261028 (+ 183us) server_negotiation.cc:299] Negotiation successful
0929 10:54:03.261045 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 10:54:03.267329 8880 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:54:03.268041 8880 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:54:03.279968 8880 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f9085e2c612465ba27c45ede7ffb173: Generated new TSK 0
I20260929 10:54:03.280193 8880 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:54:03.283226 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 8800
W20260929 10:54:03.287607 8785 connection.cc:535] client connection to 127.0.0.1:35193 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35193): Connection reset by peer (error 104)
2026-09-29T10:54:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1323 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:54:03 dist-test-slave-vqch krb5kdc[8900](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:54:03 dist-test-slave-vqch krb5kdc[8900](info): set up 2 sockets
Sep 29 10:54:03 dist-test-slave-vqch krb5kdc[8900](info): commencing operation
krb5kdc: starting...
W20260929 10:54:04.324903 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:54:04 dist-test-slave-vqch krb5kdc[8900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:54:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:54:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 10:54:04.444499 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38737
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38737
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:54:04.525305 8913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:54:04.525547 8913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:54:04.525583 8913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:54:04.526743 8913 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:54:04.526814 8913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:54:04.526837 8913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:54:04.526858 8913 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:54:04.526887 8913 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:54:04.528364 8913 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38737
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:54:04.528647 8913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:54:04.528877 8913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:54:04.533636 8919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:04.533625 8918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:04.533828 8913 server_base.cc:1034] running on GCE node
W20260929 10:54:04.533620 8921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:04.534139 8913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:54:04.534435 8913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:54:04.535604 8913 hybrid_clock.cc:648] HybridClock initialized: now 1790679244535595 us; error 36 us; skew 500 ppm
Sep 29 10:54:04 dist-test-slave-vqch krb5kdc[8900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679244, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:54:04.538080 8913 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 10:54:04.538542 8913 webserver.cc:460] Webserver started at http://127.0.0.1:43785/ using document root <none> and password file <none>
I20260929 10:54:04.538815 8913 fs_manager.cc:359] Metadata directory not provided
I20260929 10:54:04.538888 8913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:54:04.539034 8913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:54:04.540072 8913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "214b9bfe1133401fb8c8bba5a3225787"
format_stamp: "Formatted at 2026-09-29 10:54:04 on dist-test-slave-vqch"
server_key: "88fe14abb55a7224197d98ca9f35bcbb"
server_key_iv: "ab3172403c84e358a75fc66c63c252a7"
server_key_version: "encryptionkey@0"
I20260929 10:54:04.540445 8913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "214b9bfe1133401fb8c8bba5a3225787"
format_stamp: "Formatted at 2026-09-29 10:54:04 on dist-test-slave-vqch"
server_key: "88fe14abb55a7224197d98ca9f35bcbb"
server_key_iv: "ab3172403c84e358a75fc66c63c252a7"
server_key_version: "encryptionkey@0"
I20260929 10:54:04.541846 8913 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:54:04.542596 8928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:04.542832 8913 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:54:04.542937 8913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "214b9bfe1133401fb8c8bba5a3225787"
format_stamp: "Formatted at 2026-09-29 10:54:04 on dist-test-slave-vqch"
server_key: "88fe14abb55a7224197d98ca9f35bcbb"
server_key_iv: "ab3172403c84e358a75fc66c63c252a7"
server_key_version: "encryptionkey@0"
I20260929 10:54:04.543015 8913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:54:04.546450 8913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:54:04.548260 8913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:54:04.548415 8913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:54:04.552428 8913 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38737
I20260929 10:54:04.552474 8980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38737 every 8 connection(s)
I20260929 10:54:04.552819 8913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:54:04.553716 8981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:54:04.555922 8981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 214b9bfe1133401fb8c8bba5a3225787: Bootstrap starting.
I20260929 10:54:04.556497 8981 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 214b9bfe1133401fb8c8bba5a3225787: Neither blocks nor log segments found. Creating new log.
I20260929 10:54:04.556784 8981 log.cc:824] T 00000000000000000000000000000000 P 214b9bfe1133401fb8c8bba5a3225787: Log is configured to *not* fsync() on all Append() calls
I20260929 10:54:04.557585 8981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 214b9bfe1133401fb8c8bba5a3225787: No bootstrap required, opened a new log
I20260929 10:54:04.559190 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 8913
I20260929 10:54:04.559321 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:54:04.559487 20803 external_mini_cluster.cc:1428] Setting key a2d43e819f70580e3357b2e0b51f9691
I20260929 10:54:04.559510 8981 raft_consensus.cc:348] T 00000000000000000000000000000000 P 214b9bfe1133401fb8c8bba5a3225787 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "214b9bfe1133401fb8c8bba5a3225787" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38737 } }
I20260929 10:54:04.559716 8981 raft_consensus.cc:374] T 00000000000000000000000000000000 P 214b9bfe1133401fb8c8bba5a3225787 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:54:04.559768 8981 raft_consensus.cc:729] T 00000000000000000000000000000000 P 214b9bfe1133401fb8c8bba5a3225787 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 214b9bfe1133401fb8c8bba5a3225787, State: Initialized, Role: FOLLOWER
I20260929 10:54:04.559921 8981 consensus_queue.cc:260] T 00000000000000000000000000000000 P 214b9bfe1133401fb8c8bba5a3225787 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "214b9bfe1133401fb8c8bba5a3225787" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38737 } }
I20260929 10:54:04.560021 8981 raft_consensus.cc:388] T 00000000000000000000000000000000 P 214b9bfe1133401fb8c8bba5a3225787 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:54:04.560082 8981 raft_consensus.cc:482] T 00000000000000000000000000000000 P 214b9bfe1133401fb8c8bba5a3225787 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:54:04.560135 8981 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 214b9bfe1133401fb8c8bba5a3225787 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:54:04.561271 8981 raft_consensus.cc:504] T 00000000000000000000000000000000 P 214b9bfe1133401fb8c8bba5a3225787 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "214b9bfe1133401fb8c8bba5a3225787" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38737 } }
I20260929 10:54:04.561393 8981 leader_election.cc:302] T 00000000000000000000000000000000 P 214b9bfe1133401fb8c8bba5a3225787 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 214b9bfe1133401fb8c8bba5a3225787; no voters:
I20260929 10:54:04.561558 8981 leader_election.cc:288] T 00000000000000000000000000000000 P 214b9bfe1133401fb8c8bba5a3225787 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:54:04.561594 8987 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 214b9bfe1133401fb8c8bba5a3225787 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:54:04.561761 8987 raft_consensus.cc:686] T 00000000000000000000000000000000 P 214b9bfe1133401fb8c8bba5a3225787 [term 1 LEADER]: Becoming Leader. State: Replica: 214b9bfe1133401fb8c8bba5a3225787, State: Running, Role: LEADER
I20260929 10:54:04.561854 8981 sys_catalog.cc:564] T 00000000000000000000000000000000 P 214b9bfe1133401fb8c8bba5a3225787 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:54:04.561898 8986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:04.560125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55125 (local address 127.0.0.1:38737)
0929 10:54:04.560304 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:04.560313 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:04.560346 (+ 33us) server_negotiation.cc:407] Connection header received
0929 10:54:04.560466 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:04.560474 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:04.560522 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:04.560701 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:04.561452 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:04.561456 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:04.561456 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:54:04.561461 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:04.561510 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:54:04.561525 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:04.561587 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:04.561666 (+ 79us) server_negotiation.cc:299] Negotiation successful
0929 10:54:04.561724 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":91,"threads_started":1}
I20260929 10:54:04.561887 8987 consensus_queue.cc:237] T 00000000000000000000000000000000 P 214b9bfe1133401fb8c8bba5a3225787 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "214b9bfe1133401fb8c8bba5a3225787" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38737 } }
I20260929 10:54:04.562389 8988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 214b9bfe1133401fb8c8bba5a3225787 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "214b9bfe1133401fb8c8bba5a3225787" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "214b9bfe1133401fb8c8bba5a3225787" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38737 } } }
I20260929 10:54:04.562410 8989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 214b9bfe1133401fb8c8bba5a3225787 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 214b9bfe1133401fb8c8bba5a3225787. Latest consensus state: current_term: 1 leader_uuid: "214b9bfe1133401fb8c8bba5a3225787" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "214b9bfe1133401fb8c8bba5a3225787" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38737 } } }
I20260929 10:54:04.562551 8989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 214b9bfe1133401fb8c8bba5a3225787 [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:04.562484 8988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 214b9bfe1133401fb8c8bba5a3225787 [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:04.562783 8996 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:54:04.563117 20803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:54:04.563334 8996 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:54:04.565761 8986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:04.564691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54019 (local address 127.0.0.1:38737)
0929 10:54:04.564728 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:04.564729 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:04.564795 (+ 66us) server_negotiation.cc:407] Connection header received
0929 10:54:04.564859 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:04.564859 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:04.564862 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:04.564902 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:04.565491 (+ 589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:04.565493 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:04.565493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:54:04.565497 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:04.565510 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:54:04.565515 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:04.565571 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:04.565676 (+ 105us) server_negotiation.cc:299] Negotiation successful
0929 10:54:04.565692 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 10:54:04.566075 8996 catalog_manager.cc:1347] Generated new cluster ID: 3f3f8d2fcfed4542bb5b8029ecca98d0
I20260929 10:54:04.566121 8996 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:54:04.581264 8996 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:54:04.581446 8996 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:54:04.586500 8996 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 214b9bfe1133401fb8c8bba5a3225787: Generated new TSK 0
I20260929 10:54:04.586685 8996 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:54:04.590533 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 8913
2026-09-29T10:54:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1306 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:54:04 dist-test-slave-vqch krb5kdc[9013](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:54:04 dist-test-slave-vqch krb5kdc[9013](info): set up 2 sockets
Sep 29 10:54:04 dist-test-slave-vqch krb5kdc[9013](info): commencing operation
krb5kdc: starting...
W20260929 10:54:05.628906 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:54:05 dist-test-slave-vqch krb5kdc[9013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679245, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:54:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:54:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 10:54:05.751533 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37985
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37985
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:54:05.834138 9026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:54:05.834352 9026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:54:05.834378 9026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:54:05.835479 9026 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:54:05.835515 9026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:54:05.835528 9026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:54:05.835542 9026 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:54:05.835554 9026 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:54:05.837056 9026 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37985
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:54:05.837293 9026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:54:05.837481 9026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:54:05.842373 9031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:05.842344 9032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:05.842344 9034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:05.842597 9026 server_base.cc:1034] running on GCE node
I20260929 10:54:05.842864 9026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:54:05.843179 9026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:54:05.844355 9026 hybrid_clock.cc:648] HybridClock initialized: now 1790679245844344 us; error 50 us; skew 500 ppm
Sep 29 10:54:05 dist-test-slave-vqch krb5kdc[9013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679245, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:54:05.847002 9026 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 10:54:05.847419 9026 webserver.cc:460] Webserver started at http://127.0.0.1:34793/ using document root <none> and password file <none>
I20260929 10:54:05.847632 9026 fs_manager.cc:359] Metadata directory not provided
I20260929 10:54:05.847675 9026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:54:05.847764 9026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:54:05.848781 9026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "5b4065fdb8a8474ba043d14238f08813"
format_stamp: "Formatted at 2026-09-29 10:54:05 on dist-test-slave-vqch"
server_key: "1051de8b89a11cd2c076086cdf12f68d"
server_key_iv: "5165bd38b350e2d586f44a79a040657e"
server_key_version: "encryptionkey@0"
I20260929 10:54:05.849162 9026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "5b4065fdb8a8474ba043d14238f08813"
format_stamp: "Formatted at 2026-09-29 10:54:05 on dist-test-slave-vqch"
server_key: "1051de8b89a11cd2c076086cdf12f68d"
server_key_iv: "5165bd38b350e2d586f44a79a040657e"
server_key_version: "encryptionkey@0"
I20260929 10:54:05.850565 9026 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:54:05.851419 9041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:05.851621 9026 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:54:05.851706 9026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "5b4065fdb8a8474ba043d14238f08813"
format_stamp: "Formatted at 2026-09-29 10:54:05 on dist-test-slave-vqch"
server_key: "1051de8b89a11cd2c076086cdf12f68d"
server_key_iv: "5165bd38b350e2d586f44a79a040657e"
server_key_version: "encryptionkey@0"
I20260929 10:54:05.851773 9026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:54:05.867163 9026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:54:05.869114 9026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:54:05.869235 9026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:54:05.873358 9026 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37985
I20260929 10:54:05.873413 9093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37985 every 8 connection(s)
I20260929 10:54:05.873777 9026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:54:05.874681 9094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:54:05.876590 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 9026
I20260929 10:54:05.876714 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:54:05.876912 20803 external_mini_cluster.cc:1428] Setting key 3a7bf4a1a38b36f8ea5c2246f538dca7
I20260929 10:54:05.877313 9094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b4065fdb8a8474ba043d14238f08813: Bootstrap starting.
I20260929 10:54:05.878233 9094 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b4065fdb8a8474ba043d14238f08813: Neither blocks nor log segments found. Creating new log.
I20260929 10:54:05.878521 9094 log.cc:824] T 00000000000000000000000000000000 P 5b4065fdb8a8474ba043d14238f08813: Log is configured to *not* fsync() on all Append() calls
I20260929 10:54:05.879221 9094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b4065fdb8a8474ba043d14238f08813: No bootstrap required, opened a new log
I20260929 10:54:05.881049 9094 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b4065fdb8a8474ba043d14238f08813 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b4065fdb8a8474ba043d14238f08813" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37985 } }
I20260929 10:54:05.881213 9094 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b4065fdb8a8474ba043d14238f08813 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:54:05.881258 9094 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b4065fdb8a8474ba043d14238f08813 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b4065fdb8a8474ba043d14238f08813, State: Initialized, Role: FOLLOWER
I20260929 10:54:05.881407 9094 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b4065fdb8a8474ba043d14238f08813 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b4065fdb8a8474ba043d14238f08813" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37985 } }
I20260929 10:54:05.881510 9094 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b4065fdb8a8474ba043d14238f08813 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:54:05.881541 9094 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b4065fdb8a8474ba043d14238f08813 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:54:05.881618 9094 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b4065fdb8a8474ba043d14238f08813 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:54:05.882351 9094 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b4065fdb8a8474ba043d14238f08813 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b4065fdb8a8474ba043d14238f08813" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37985 } }
I20260929 10:54:05.882473 9094 leader_election.cc:302] T 00000000000000000000000000000000 P 5b4065fdb8a8474ba043d14238f08813 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b4065fdb8a8474ba043d14238f08813; no voters:
I20260929 10:54:05.882658 9094 leader_election.cc:288] T 00000000000000000000000000000000 P 5b4065fdb8a8474ba043d14238f08813 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:54:05.882686 9099 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b4065fdb8a8474ba043d14238f08813 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:54:05.882874 9099 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b4065fdb8a8474ba043d14238f08813 [term 1 LEADER]: Becoming Leader. State: Replica: 5b4065fdb8a8474ba043d14238f08813, State: Running, Role: LEADER
I20260929 10:54:05.882925 9094 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b4065fdb8a8474ba043d14238f08813 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:54:05.883015 9099 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b4065fdb8a8474ba043d14238f08813 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b4065fdb8a8474ba043d14238f08813" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37985 } }
I20260929 10:54:05.883373 9099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b4065fdb8a8474ba043d14238f08813 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b4065fdb8a8474ba043d14238f08813. Latest consensus state: current_term: 1 leader_uuid: "5b4065fdb8a8474ba043d14238f08813" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b4065fdb8a8474ba043d14238f08813" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37985 } } }
I20260929 10:54:05.883597 9099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b4065fdb8a8474ba043d14238f08813 [sys.catalog]: This master's current role is: LEADER
Sep 29 10:54:05 dist-test-slave-vqch krb5kdc[9013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679245, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 10:54:05.883508 9100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b4065fdb8a8474ba043d14238f08813 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b4065fdb8a8474ba043d14238f08813" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b4065fdb8a8474ba043d14238f08813" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37985 } } }
I20260929 10:54:05.883972 9100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b4065fdb8a8474ba043d14238f08813 [sys.catalog]: This master's current role is: LEADER
W20260929 10:54:05.884699 9115 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5b4065fdb8a8474ba043d14238f08813: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:54:05.884765 9115 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:54:05.884814 9108 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:54:05.885079 9108 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:54:05.886765 9108 catalog_manager.cc:1347] Generated new cluster ID: 0a1d78dbfc5e4cc38b930ea88acff02d
I20260929 10:54:05.886807 9108 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:54:05.888659 9097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:05.877519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37481 (local address 127.0.0.1:37985)
0929 10:54:05.877712 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:05.877721 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:05.877750 (+ 29us) server_negotiation.cc:407] Connection header received
0929 10:54:05.878176 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:05.878182 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:05.878226 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:05.878402 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:05.879120 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:05.880053 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:05.880836 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:05.881374 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:05.881485 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:05.884672 (+ 3187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:05.884677 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:05.884677 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:05.884688 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:05.887048 (+ 2360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:05.887457 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:05.887460 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:05.887460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:05.887558 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:05.887782 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:05.887783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:05.887783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:05.888059 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:05.888203 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:05.888340 (+ 137us) server_negotiation.cc:299] Negotiation successful
0929 10:54:05.888464 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":98,"threads_started":1}
I20260929 10:54:05.889601 20803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:54:05.899669 9097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:05.891207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41583 (local address 127.0.0.1:37985)
0929 10:54:05.891244 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:05.891246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:05.891379 (+ 133us) server_negotiation.cc:407] Connection header received
0929 10:54:05.891434 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:05.891435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:05.891445 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:05.891494 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:05.892117 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:05.892897 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:05.894896 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:05.895232 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:05.895277 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:05.896826 (+ 1549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:05.896827 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:05.896828 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:05.896836 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:05.898422 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:05.898832 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:05.898833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:05.898833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:05.898890 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:05.899085 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:05.899085 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:05.899086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:05.899211 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:05.899283 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:05.899481 (+ 198us) server_negotiation.cc:299] Negotiation successful
0929 10:54:05.899564 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 10:54:05.902413 9108 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:54:05.903141 9108 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:54:05.916528 9108 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b4065fdb8a8474ba043d14238f08813: Generated new TSK 0
I20260929 10:54:05.916770 9108 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:54:05.921454 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 9026
2026-09-29T10:54:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1332 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:54:05 dist-test-slave-vqch krb5kdc[9126](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:54:05 dist-test-slave-vqch krb5kdc[9126](info): set up 2 sockets
Sep 29 10:54:05 dist-test-slave-vqch krb5kdc[9126](info): commencing operation
krb5kdc: starting...
W20260929 10:54:06.960842 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:54:07 dist-test-slave-vqch krb5kdc[9126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:54:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:54:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 10:54:07.079635 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45115
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:54:07.160542 9139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:54:07.160784 9139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:54:07.160820 9139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:54:07.161988 9139 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:54:07.162053 9139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:54:07.162077 9139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:54:07.162124 9139 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:54:07.162153 9139 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:54:07.163578 9139 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45115
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:54:07.163863 9139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:54:07.164077 9139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:54:07.168658 9145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:07.168668 9144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:07.168915 9139 server_base.cc:1034] running on GCE node
W20260929 10:54:07.169070 9147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:07.169308 9139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:54:07.169544 9139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:54:07.170686 9139 hybrid_clock.cc:648] HybridClock initialized: now 1790679247170672 us; error 29 us; skew 500 ppm
Sep 29 10:54:07 dist-test-slave-vqch krb5kdc[9126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679247, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:54:07.178612 9139 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 10:54:07.179004 9139 webserver.cc:460] Webserver started at http://127.0.0.1:35263/ using document root <none> and password file <none>
I20260929 10:54:07.179193 9139 fs_manager.cc:359] Metadata directory not provided
I20260929 10:54:07.179248 9139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:54:07.179335 9139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:54:07.180315 9139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "25851b1a09ca4709b0bd8ef0418680ca"
format_stamp: "Formatted at 2026-09-29 10:54:07 on dist-test-slave-vqch"
server_key: "e44b9e2f2b6cb388fdb647eb393da146"
server_key_iv: "4e91ab96f04b25614261974ab9e1bd7b"
server_key_version: "encryptionkey@0"
I20260929 10:54:07.180611 9139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "25851b1a09ca4709b0bd8ef0418680ca"
format_stamp: "Formatted at 2026-09-29 10:54:07 on dist-test-slave-vqch"
server_key: "e44b9e2f2b6cb388fdb647eb393da146"
server_key_iv: "4e91ab96f04b25614261974ab9e1bd7b"
server_key_version: "encryptionkey@0"
I20260929 10:54:07.181948 9139 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 10:54:07.182626 9154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:07.182804 9139 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:54:07.182909 9139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "25851b1a09ca4709b0bd8ef0418680ca"
format_stamp: "Formatted at 2026-09-29 10:54:07 on dist-test-slave-vqch"
server_key: "e44b9e2f2b6cb388fdb647eb393da146"
server_key_iv: "4e91ab96f04b25614261974ab9e1bd7b"
server_key_version: "encryptionkey@0"
I20260929 10:54:07.182983 9139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:54:07.202793 9139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:54:07.204643 9139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:54:07.204802 9139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:54:07.208724 9139 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45115
I20260929 10:54:07.208762 9206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45115 every 8 connection(s)
I20260929 10:54:07.209177 9139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:54:07.210086 9207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:54:07.212361 9207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25851b1a09ca4709b0bd8ef0418680ca: Bootstrap starting.
I20260929 10:54:07.212944 9207 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25851b1a09ca4709b0bd8ef0418680ca: Neither blocks nor log segments found. Creating new log.
I20260929 10:54:07.213248 9207 log.cc:824] T 00000000000000000000000000000000 P 25851b1a09ca4709b0bd8ef0418680ca: Log is configured to *not* fsync() on all Append() calls
I20260929 10:54:07.213924 9207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25851b1a09ca4709b0bd8ef0418680ca: No bootstrap required, opened a new log
I20260929 10:54:07.214721 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 9139
I20260929 10:54:07.214854 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:54:07.215054 20803 external_mini_cluster.cc:1428] Setting key ce61b405014699a2d79c6dc113178b6c
I20260929 10:54:07.215909 9207 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25851b1a09ca4709b0bd8ef0418680ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25851b1a09ca4709b0bd8ef0418680ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45115 } }
I20260929 10:54:07.216078 9207 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25851b1a09ca4709b0bd8ef0418680ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:54:07.216118 9207 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25851b1a09ca4709b0bd8ef0418680ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25851b1a09ca4709b0bd8ef0418680ca, State: Initialized, Role: FOLLOWER
I20260929 10:54:07.216250 9207 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25851b1a09ca4709b0bd8ef0418680ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25851b1a09ca4709b0bd8ef0418680ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45115 } }
I20260929 10:54:07.216341 9207 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25851b1a09ca4709b0bd8ef0418680ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:54:07.216365 9207 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25851b1a09ca4709b0bd8ef0418680ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:54:07.216434 9207 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25851b1a09ca4709b0bd8ef0418680ca [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:54:07.216964 9207 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25851b1a09ca4709b0bd8ef0418680ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25851b1a09ca4709b0bd8ef0418680ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45115 } }
I20260929 10:54:07.217110 9207 leader_election.cc:302] T 00000000000000000000000000000000 P 25851b1a09ca4709b0bd8ef0418680ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25851b1a09ca4709b0bd8ef0418680ca; no voters:
I20260929 10:54:07.217281 9207 leader_election.cc:288] T 00000000000000000000000000000000 P 25851b1a09ca4709b0bd8ef0418680ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:54:07.217306 9212 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25851b1a09ca4709b0bd8ef0418680ca [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:54:07.217411 9212 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25851b1a09ca4709b0bd8ef0418680ca [term 1 LEADER]: Becoming Leader. State: Replica: 25851b1a09ca4709b0bd8ef0418680ca, State: Running, Role: LEADER
I20260929 10:54:07.217533 9207 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25851b1a09ca4709b0bd8ef0418680ca [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:54:07.217550 9212 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25851b1a09ca4709b0bd8ef0418680ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25851b1a09ca4709b0bd8ef0418680ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45115 } }
I20260929 10:54:07.217986 9212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25851b1a09ca4709b0bd8ef0418680ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25851b1a09ca4709b0bd8ef0418680ca. Latest consensus state: current_term: 1 leader_uuid: "25851b1a09ca4709b0bd8ef0418680ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25851b1a09ca4709b0bd8ef0418680ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45115 } } }
I20260929 10:54:07.218222 9212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25851b1a09ca4709b0bd8ef0418680ca [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:07.218151 9213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25851b1a09ca4709b0bd8ef0418680ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25851b1a09ca4709b0bd8ef0418680ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25851b1a09ca4709b0bd8ef0418680ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45115 } } }
I20260929 10:54:07.218657 9221 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:54:07.219240 9221 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:54:07.218663 9213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25851b1a09ca4709b0bd8ef0418680ca [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:07.221123 9211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:07.215569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50033 (local address 127.0.0.1:45115)
0929 10:54:07.215747 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:07.215763 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:07.215788 (+ 25us) server_negotiation.cc:407] Connection header received
0929 10:54:07.215882 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:07.215887 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:07.215928 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:07.216033 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:07.216278 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:07.217804 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:07.218934 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:07.219485 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:07.219612 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:07.220345 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:07.220348 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:07.220349 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:54:07.220547 (+ 198us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:07.220602 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:54:07.220617 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:07.220675 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:07.220785 (+ 110us) server_negotiation.cc:299] Negotiation successful
0929 10:54:07.220852 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":97,"threads_started":1}
I20260929 10:54:07.221698 9221 catalog_manager.cc:1347] Generated new cluster ID: d4b34b5eb2f74d3ba2ee324acea7d20a
I20260929 10:54:07.221784 9221 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:54:07.222196 20803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:54:07.227026 9211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:07.223598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47477 (local address 127.0.0.1:45115)
0929 10:54:07.223631 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:07.223632 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:07.223794 (+ 162us) server_negotiation.cc:407] Connection header received
0929 10:54:07.223824 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:07.223825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:07.223835 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:07.223899 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:07.224001 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:07.224707 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:07.225683 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:07.226052 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:07.226143 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:07.226793 (+ 650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:07.226795 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:07.226795 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:54:07.226799 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:07.226815 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:54:07.226819 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:07.226879 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:07.226934 (+ 55us) server_negotiation.cc:299] Negotiation successful
0929 10:54:07.226956 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 10:54:07.229621 9221 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:54:07.230114 9221 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:54:07.234652 9221 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25851b1a09ca4709b0bd8ef0418680ca: Generated new TSK 0
I20260929 10:54:07.234835 9221 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:54:07.236929 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 9139
2026-09-29T10:54:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1314 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:54:07 dist-test-slave-vqch krb5kdc[9238](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:54:07 dist-test-slave-vqch krb5kdc[9238](info): set up 2 sockets
Sep 29 10:54:07 dist-test-slave-vqch krb5kdc[9238](info): commencing operation
krb5kdc: starting...
W20260929 10:54:08.272730 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:54:08 dist-test-slave-vqch krb5kdc[9238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679248, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:54:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:54:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 10:54:08.389780 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36613
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36613
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:54:08.470417 9251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:54:08.470657 9251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:54:08.470705 9251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:54:08.471881 9251 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:54:08.471951 9251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:54:08.471976 9251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:54:08.472003 9251 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:54:08.472035 9251 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:54:08.473577 9251 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36613
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:54:08.473862 9251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:54:08.474090 9251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:54:08.479058 9257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:08.479068 9256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:08.479246 9251 server_base.cc:1034] running on GCE node
W20260929 10:54:08.479059 9259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:08.479532 9251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:54:08.479846 9251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:54:08.481034 9251 hybrid_clock.cc:648] HybridClock initialized: now 1790679248481011 us; error 37 us; skew 500 ppm
Sep 29 10:54:08 dist-test-slave-vqch krb5kdc[9238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679248, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:54:08.483723 9251 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 10:54:08.484212 9251 webserver.cc:460] Webserver started at http://127.0.0.1:44359/ using document root <none> and password file <none>
I20260929 10:54:08.484448 9251 fs_manager.cc:359] Metadata directory not provided
I20260929 10:54:08.484517 9251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:54:08.484666 9251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:54:08.485795 9251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "8c8b480387f74c0f8cecf0535bdc515d"
format_stamp: "Formatted at 2026-09-29 10:54:08 on dist-test-slave-vqch"
server_key: "0f194acf06b2d0ebd6bf30b667a3b9f7"
server_key_iv: "128dd4a1135aed3b1dddab3eeefb4fc0"
server_key_version: "encryptionkey@0"
I20260929 10:54:08.486158 9251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "8c8b480387f74c0f8cecf0535bdc515d"
format_stamp: "Formatted at 2026-09-29 10:54:08 on dist-test-slave-vqch"
server_key: "0f194acf06b2d0ebd6bf30b667a3b9f7"
server_key_iv: "128dd4a1135aed3b1dddab3eeefb4fc0"
server_key_version: "encryptionkey@0"
I20260929 10:54:08.487499 9251 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 10:54:08.488279 9266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:08.488453 9251 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:54:08.488550 9251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "8c8b480387f74c0f8cecf0535bdc515d"
format_stamp: "Formatted at 2026-09-29 10:54:08 on dist-test-slave-vqch"
server_key: "0f194acf06b2d0ebd6bf30b667a3b9f7"
server_key_iv: "128dd4a1135aed3b1dddab3eeefb4fc0"
server_key_version: "encryptionkey@0"
I20260929 10:54:08.488631 9251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:54:08.491981 9251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:54:08.494321 9251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:54:08.494478 9251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:54:08.498627 9251 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36613
I20260929 10:54:08.498660 9318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36613 every 8 connection(s)
I20260929 10:54:08.499017 9251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:54:08.499850 9319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:54:08.502144 9319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c8b480387f74c0f8cecf0535bdc515d: Bootstrap starting.
I20260929 10:54:08.502755 9319 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c8b480387f74c0f8cecf0535bdc515d: Neither blocks nor log segments found. Creating new log.
I20260929 10:54:08.503037 9319 log.cc:824] T 00000000000000000000000000000000 P 8c8b480387f74c0f8cecf0535bdc515d: Log is configured to *not* fsync() on all Append() calls
I20260929 10:54:08.503728 9319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c8b480387f74c0f8cecf0535bdc515d: No bootstrap required, opened a new log
I20260929 10:54:08.504205 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 9251
I20260929 10:54:08.504333 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:54:08.504509 20803 external_mini_cluster.cc:1428] Setting key 253360e52c98fac1fc951a9c4d8993dd
I20260929 10:54:08.505977 9319 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c8b480387f74c0f8cecf0535bdc515d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8b480387f74c0f8cecf0535bdc515d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36613 } }
I20260929 10:54:08.506132 9319 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c8b480387f74c0f8cecf0535bdc515d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:54:08.506173 9319 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c8b480387f74c0f8cecf0535bdc515d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c8b480387f74c0f8cecf0535bdc515d, State: Initialized, Role: FOLLOWER
I20260929 10:54:08.506307 9319 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c8b480387f74c0f8cecf0535bdc515d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8b480387f74c0f8cecf0535bdc515d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36613 } }
I20260929 10:54:08.506402 9319 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c8b480387f74c0f8cecf0535bdc515d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:54:08.506438 9319 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c8b480387f74c0f8cecf0535bdc515d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:54:08.506525 9319 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c8b480387f74c0f8cecf0535bdc515d [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:54:08.507406 9323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:08.505013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55179 (local address 127.0.0.1:36613)
0929 10:54:08.505169 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:08.505176 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:08.505200 (+ 24us) server_negotiation.cc:407] Connection header received
0929 10:54:08.505969 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:08.505977 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:08.506007 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:08.506224 (+ 217us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:08.506927 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:08.506930 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:08.506930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:54:08.506934 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:08.506971 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:54:08.506981 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:08.507037 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:08.507137 (+ 100us) server_negotiation.cc:299] Negotiation successful
0929 10:54:08.507190 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":69,"threads_started":1}
I20260929 10:54:08.507579 9319 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c8b480387f74c0f8cecf0535bdc515d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8b480387f74c0f8cecf0535bdc515d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36613 } }
I20260929 10:54:08.507682 9319 leader_election.cc:302] T 00000000000000000000000000000000 P 8c8b480387f74c0f8cecf0535bdc515d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c8b480387f74c0f8cecf0535bdc515d; no voters:
I20260929 10:54:08.507839 9319 leader_election.cc:288] T 00000000000000000000000000000000 P 8c8b480387f74c0f8cecf0535bdc515d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:54:08.507901 9324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c8b480387f74c0f8cecf0535bdc515d [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:54:08.508085 9319 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c8b480387f74c0f8cecf0535bdc515d [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:54:08.508105 9324 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c8b480387f74c0f8cecf0535bdc515d [term 1 LEADER]: Becoming Leader. State: Replica: 8c8b480387f74c0f8cecf0535bdc515d, State: Running, Role: LEADER
I20260929 10:54:08.508245 9324 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c8b480387f74c0f8cecf0535bdc515d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8b480387f74c0f8cecf0535bdc515d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36613 } }
I20260929 10:54:08.508358 20803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:54:08.509531 9326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c8b480387f74c0f8cecf0535bdc515d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c8b480387f74c0f8cecf0535bdc515d. Latest consensus state: current_term: 1 leader_uuid: "8c8b480387f74c0f8cecf0535bdc515d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8b480387f74c0f8cecf0535bdc515d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36613 } } }
I20260929 10:54:08.509639 9326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c8b480387f74c0f8cecf0535bdc515d [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:08.509778 9325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c8b480387f74c0f8cecf0535bdc515d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c8b480387f74c0f8cecf0535bdc515d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8b480387f74c0f8cecf0535bdc515d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36613 } } }
I20260929 10:54:08.509999 9325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c8b480387f74c0f8cecf0535bdc515d [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:08.510267 9344 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:54:08.510763 9344 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:54:08.512084 9323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:08.510066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55753 (local address 127.0.0.1:36613)
0929 10:54:08.510119 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:08.510120 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:08.510234 (+ 114us) server_negotiation.cc:407] Connection header received
0929 10:54:08.510305 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:08.510306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:08.510310 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:08.510385 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:08.511843 (+ 1458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:08.511844 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:08.511844 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:54:08.511849 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:08.511862 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:54:08.511866 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:08.511922 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:08.511979 (+ 57us) server_negotiation.cc:299] Negotiation successful
0929 10:54:08.511993 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 10:54:08.512281 9344 catalog_manager.cc:1347] Generated new cluster ID: 787e397876c14ceeb84b49e6cc5381ee
I20260929 10:54:08.512348 9344 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:54:08.523366 9344 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:54:08.523536 9344 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:54:08.533924 9344 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c8b480387f74c0f8cecf0535bdc515d: Generated new TSK 0
I20260929 10:54:08.534157 9344 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:54:08.537881 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 9251
2026-09-29T10:54:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1300 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:54:08 dist-test-slave-vqch krb5kdc[9350](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:54:08 dist-test-slave-vqch krb5kdc[9350](info): set up 2 sockets
Sep 29 10:54:08 dist-test-slave-vqch krb5kdc[9350](info): commencing operation
krb5kdc: starting...
W20260929 10:54:09.589538 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:54:09 dist-test-slave-vqch krb5kdc[9350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:54:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:54:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 10:54:09.708351 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35257
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35257
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:54:09.788717 9363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:54:09.788967 9363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:54:09.789021 9363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:54:09.790192 9363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:54:09.790266 9363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:54:09.790292 9363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:54:09.790323 9363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:54:09.790347 9363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:54:09.791826 9363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:54:09.792100 9363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:54:09.792326 9363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:54:09.797490 9369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:09.797456 9371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:09.797631 9363 server_base.cc:1034] running on GCE node
W20260929 10:54:09.797456 9368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:09.798053 9363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:54:09.798321 9363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:54:09.799505 9363 hybrid_clock.cc:648] HybridClock initialized: now 1790679249799491 us; error 46 us; skew 500 ppm
Sep 29 10:54:09 dist-test-slave-vqch krb5kdc[9350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679249, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:54:09.802187 9363 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 10:54:09.802671 9363 webserver.cc:460] Webserver started at http://127.0.0.1:33767/ using document root <none> and password file <none>
I20260929 10:54:09.802933 9363 fs_manager.cc:359] Metadata directory not provided
I20260929 10:54:09.803001 9363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:54:09.803157 9363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:54:09.804204 9363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "30ea46713588416bb2cda5fd17ba17c8"
format_stamp: "Formatted at 2026-09-29 10:54:09 on dist-test-slave-vqch"
server_key: "a19ddc26cb3e2d80a6f2bd1bbbdb02fe"
server_key_iv: "f82e6a3c2f6c50d3667572b7f3054a50"
server_key_version: "encryptionkey@0"
I20260929 10:54:09.804577 9363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "30ea46713588416bb2cda5fd17ba17c8"
format_stamp: "Formatted at 2026-09-29 10:54:09 on dist-test-slave-vqch"
server_key: "a19ddc26cb3e2d80a6f2bd1bbbdb02fe"
server_key_iv: "f82e6a3c2f6c50d3667572b7f3054a50"
server_key_version: "encryptionkey@0"
I20260929 10:54:09.805997 9363 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 10:54:09.806726 9379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:09.806910 9363 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:54:09.807004 9363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "30ea46713588416bb2cda5fd17ba17c8"
format_stamp: "Formatted at 2026-09-29 10:54:09 on dist-test-slave-vqch"
server_key: "a19ddc26cb3e2d80a6f2bd1bbbdb02fe"
server_key_iv: "f82e6a3c2f6c50d3667572b7f3054a50"
server_key_version: "encryptionkey@0"
I20260929 10:54:09.807085 9363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:54:09.822731 9363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:54:09.825110 9363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:54:09.825280 9363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:54:09.829169 9363 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35257
I20260929 10:54:09.829216 9432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35257 every 8 connection(s)
I20260929 10:54:09.829610 9363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:54:09.830618 9433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:54:09.832952 9433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30ea46713588416bb2cda5fd17ba17c8: Bootstrap starting.
I20260929 10:54:09.833094 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 9363
I20260929 10:54:09.833225 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:54:09.833400 20803 external_mini_cluster.cc:1428] Setting key 8bb7f60ce11407aa8cd8973191f128d4
I20260929 10:54:09.833722 9433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30ea46713588416bb2cda5fd17ba17c8: Neither blocks nor log segments found. Creating new log.
I20260929 10:54:09.834065 9433 log.cc:824] T 00000000000000000000000000000000 P 30ea46713588416bb2cda5fd17ba17c8: Log is configured to *not* fsync() on all Append() calls
I20260929 10:54:09.834834 9433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30ea46713588416bb2cda5fd17ba17c8: No bootstrap required, opened a new log
I20260929 10:54:09.837185 9433 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30ea46713588416bb2cda5fd17ba17c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30ea46713588416bb2cda5fd17ba17c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35257 } }
I20260929 10:54:09.837361 9433 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30ea46713588416bb2cda5fd17ba17c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:54:09.837389 9433 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30ea46713588416bb2cda5fd17ba17c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30ea46713588416bb2cda5fd17ba17c8, State: Initialized, Role: FOLLOWER
I20260929 10:54:09.837534 9433 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30ea46713588416bb2cda5fd17ba17c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30ea46713588416bb2cda5fd17ba17c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35257 } }
I20260929 10:54:09.837607 9433 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30ea46713588416bb2cda5fd17ba17c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:54:09.837630 9433 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30ea46713588416bb2cda5fd17ba17c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:54:09.837702 9433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30ea46713588416bb2cda5fd17ba17c8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:54:09.838322 9433 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30ea46713588416bb2cda5fd17ba17c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30ea46713588416bb2cda5fd17ba17c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35257 } }
I20260929 10:54:09.838428 9433 leader_election.cc:302] T 00000000000000000000000000000000 P 30ea46713588416bb2cda5fd17ba17c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30ea46713588416bb2cda5fd17ba17c8; no voters:
I20260929 10:54:09.838575 9433 leader_election.cc:288] T 00000000000000000000000000000000 P 30ea46713588416bb2cda5fd17ba17c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:54:09.838619 9438 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30ea46713588416bb2cda5fd17ba17c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:54:09.838807 9438 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30ea46713588416bb2cda5fd17ba17c8 [term 1 LEADER]: Becoming Leader. State: Replica: 30ea46713588416bb2cda5fd17ba17c8, State: Running, Role: LEADER
I20260929 10:54:09.838826 9433 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30ea46713588416bb2cda5fd17ba17c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:54:09.839900 9438 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30ea46713588416bb2cda5fd17ba17c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30ea46713588416bb2cda5fd17ba17c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35257 } }
I20260929 10:54:09.840631 9439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30ea46713588416bb2cda5fd17ba17c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30ea46713588416bb2cda5fd17ba17c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30ea46713588416bb2cda5fd17ba17c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35257 } } }
I20260929 10:54:09.840711 9439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30ea46713588416bb2cda5fd17ba17c8 [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:09.840896 9440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30ea46713588416bb2cda5fd17ba17c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30ea46713588416bb2cda5fd17ba17c8. Latest consensus state: current_term: 1 leader_uuid: "30ea46713588416bb2cda5fd17ba17c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30ea46713588416bb2cda5fd17ba17c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35257 } } }
I20260929 10:54:09.840965 9440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30ea46713588416bb2cda5fd17ba17c8 [sys.catalog]: This master's current role is: LEADER
W20260929 10:54:09.841185 9452 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 30ea46713588416bb2cda5fd17ba17c8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:54:09.841287 9452 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Sep 29 10:54:09 dist-test-slave-vqch krb5kdc[9350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 10:54:09.841370 9453 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:54:09.841797 9453 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:54:09.843550 9453 catalog_manager.cc:1347] Generated new cluster ID: e067744b25e841188a62a0978f772bf6
I20260929 10:54:09.843605 9453 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:54:09.845866 9436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:09.833969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:47693 (local address 127.0.0.1:35257)
0929 10:54:09.834187 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:09.834196 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:09.834232 (+ 36us) server_negotiation.cc:407] Connection header received
0929 10:54:09.834812 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:09.834820 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:09.834886 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:09.835040 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:09.835632 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:09.837145 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:09.838106 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:09.838746 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:09.838873 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:09.841888 (+ 3015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:09.841892 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:09.841892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:09.841903 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:09.843921 (+ 2018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:09.844394 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:09.844397 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:09.844398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:09.844525 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:09.844802 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:09.844803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:09.844804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:09.845205 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:09.845406 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:09.845501 (+ 95us) server_negotiation.cc:299] Negotiation successful
0929 10:54:09.845627 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":93,"threads_started":1}
I20260929 10:54:09.847010 20803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:54:09.850818 9453 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:54:09.851485 9453 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:54:09.857165 9436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:09.849561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:49849 (local address 127.0.0.1:35257)
0929 10:54:09.849604 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:09.849606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:09.849693 (+ 87us) server_negotiation.cc:407] Connection header received
0929 10:54:09.849744 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:09.849745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:09.849756 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:09.849811 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:09.850550 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:09.851288 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:09.852344 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:09.852650 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:09.852714 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:09.854174 (+ 1460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:09.854176 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:09.854176 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:09.854183 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:09.855808 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:09.856285 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:09.856286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:09.856287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:09.856343 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:09.856586 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:09.856587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:09.856587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:09.856704 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:09.856826 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:09.857011 (+ 185us) server_negotiation.cc:299] Negotiation successful
0929 10:54:09.857077 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 10:54:09.860237 9453 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30ea46713588416bb2cda5fd17ba17c8: Generated new TSK 0
I20260929 10:54:09.860422 9453 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:54:09.865598 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 9363
W20260929 10:54:09.869792 9348 connection.cc:535] client connection to 127.0.0.1:35257 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35257): Connection reset by peer (error 104)
2026-09-29T10:54:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1327 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:54:09 dist-test-slave-vqch krb5kdc[9464](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:54:09 dist-test-slave-vqch krb5kdc[9464](info): set up 2 sockets
Sep 29 10:54:09 dist-test-slave-vqch krb5kdc[9464](info): commencing operation
krb5kdc: starting...
W20260929 10:54:10.925046 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:54:11 dist-test-slave-vqch krb5kdc[9464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:54:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:54:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 10:54:11.044219 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46755
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46755
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:54:11.126119 9478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:54:11.126379 9478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:54:11.126415 9478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:54:11.127571 9478 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:54:11.127657 9478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:54:11.127686 9478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:54:11.127718 9478 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:54:11.127740 9478 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:54:11.129307 9478 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46755
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:54:11.129596 9478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:54:11.129835 9478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:54:11.134869 9483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:11.134862 9484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:11.134862 9486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:11.134984 9478 server_base.cc:1034] running on GCE node
I20260929 10:54:11.135303 9478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:54:11.135550 9478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:54:11.136690 9478 hybrid_clock.cc:648] HybridClock initialized: now 1790679251136663 us; error 40 us; skew 500 ppm
Sep 29 10:54:11 dist-test-slave-vqch krb5kdc[9464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679251, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:54:11.139159 9478 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 10:54:11.139618 9478 webserver.cc:460] Webserver started at http://127.0.0.1:40477/ using document root <none> and password file <none>
I20260929 10:54:11.139827 9478 fs_manager.cc:359] Metadata directory not provided
I20260929 10:54:11.139870 9478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:54:11.139961 9478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:54:11.141033 9478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "387409dfe6a149db91109fe52ea05723"
format_stamp: "Formatted at 2026-09-29 10:54:11 on dist-test-slave-vqch"
server_key: "90a29d54a196d70b9714ff4f740d0195"
server_key_iv: "d24f003fa5dbdf95d1a6b8c1e943666b"
server_key_version: "encryptionkey@0"
I20260929 10:54:11.141366 9478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "387409dfe6a149db91109fe52ea05723"
format_stamp: "Formatted at 2026-09-29 10:54:11 on dist-test-slave-vqch"
server_key: "90a29d54a196d70b9714ff4f740d0195"
server_key_iv: "d24f003fa5dbdf95d1a6b8c1e943666b"
server_key_version: "encryptionkey@0"
I20260929 10:54:11.142634 9478 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:54:11.143338 9493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:11.143537 9478 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:54:11.143615 9478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "387409dfe6a149db91109fe52ea05723"
format_stamp: "Formatted at 2026-09-29 10:54:11 on dist-test-slave-vqch"
server_key: "90a29d54a196d70b9714ff4f740d0195"
server_key_iv: "d24f003fa5dbdf95d1a6b8c1e943666b"
server_key_version: "encryptionkey@0"
I20260929 10:54:11.143710 9478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:54:11.163897 9478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:54:11.166316 9478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:54:11.166466 9478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:54:11.170656 9478 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46755
I20260929 10:54:11.170717 9545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46755 every 8 connection(s)
I20260929 10:54:11.171072 9478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:54:11.171911 9546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:54:11.174191 9546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 387409dfe6a149db91109fe52ea05723: Bootstrap starting.
I20260929 10:54:11.174706 9546 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 387409dfe6a149db91109fe52ea05723: Neither blocks nor log segments found. Creating new log.
I20260929 10:54:11.174942 9546 log.cc:824] T 00000000000000000000000000000000 P 387409dfe6a149db91109fe52ea05723: Log is configured to *not* fsync() on all Append() calls
I20260929 10:54:11.175568 9546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 387409dfe6a149db91109fe52ea05723: No bootstrap required, opened a new log
I20260929 10:54:11.177089 9546 raft_consensus.cc:348] T 00000000000000000000000000000000 P 387409dfe6a149db91109fe52ea05723 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387409dfe6a149db91109fe52ea05723" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46755 } }
I20260929 10:54:11.177219 9546 raft_consensus.cc:374] T 00000000000000000000000000000000 P 387409dfe6a149db91109fe52ea05723 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:54:11.177258 9546 raft_consensus.cc:729] T 00000000000000000000000000000000 P 387409dfe6a149db91109fe52ea05723 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 387409dfe6a149db91109fe52ea05723, State: Initialized, Role: FOLLOWER
I20260929 10:54:11.177368 9546 consensus_queue.cc:260] T 00000000000000000000000000000000 P 387409dfe6a149db91109fe52ea05723 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387409dfe6a149db91109fe52ea05723" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46755 } }
I20260929 10:54:11.177453 9546 raft_consensus.cc:388] T 00000000000000000000000000000000 P 387409dfe6a149db91109fe52ea05723 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:54:11.177488 9546 raft_consensus.cc:482] T 00000000000000000000000000000000 P 387409dfe6a149db91109fe52ea05723 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:54:11.177556 9546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 387409dfe6a149db91109fe52ea05723 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:54:11.178102 9546 raft_consensus.cc:504] T 00000000000000000000000000000000 P 387409dfe6a149db91109fe52ea05723 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387409dfe6a149db91109fe52ea05723" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46755 } }
I20260929 10:54:11.178217 9546 leader_election.cc:302] T 00000000000000000000000000000000 P 387409dfe6a149db91109fe52ea05723 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 387409dfe6a149db91109fe52ea05723; no voters:
I20260929 10:54:11.178404 9546 leader_election.cc:288] T 00000000000000000000000000000000 P 387409dfe6a149db91109fe52ea05723 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:54:11.178434 9549 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 387409dfe6a149db91109fe52ea05723 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:54:11.178601 9549 raft_consensus.cc:686] T 00000000000000000000000000000000 P 387409dfe6a149db91109fe52ea05723 [term 1 LEADER]: Becoming Leader. State: Replica: 387409dfe6a149db91109fe52ea05723, State: Running, Role: LEADER
I20260929 10:54:11.178653 9546 sys_catalog.cc:564] T 00000000000000000000000000000000 P 387409dfe6a149db91109fe52ea05723 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:54:11.178824 9549 consensus_queue.cc:237] T 00000000000000000000000000000000 P 387409dfe6a149db91109fe52ea05723 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387409dfe6a149db91109fe52ea05723" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46755 } }
I20260929 10:54:11.179193 9552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 387409dfe6a149db91109fe52ea05723 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 387409dfe6a149db91109fe52ea05723. Latest consensus state: current_term: 1 leader_uuid: "387409dfe6a149db91109fe52ea05723" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387409dfe6a149db91109fe52ea05723" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46755 } } }
I20260929 10:54:11.179299 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 9478
I20260929 10:54:11.179323 9550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 387409dfe6a149db91109fe52ea05723 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "387409dfe6a149db91109fe52ea05723" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387409dfe6a149db91109fe52ea05723" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46755 } } }
I20260929 10:54:11.179343 9552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 387409dfe6a149db91109fe52ea05723 [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:11.179374 9550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 387409dfe6a149db91109fe52ea05723 [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:11.179435 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:54:11.179580 9559 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:54:11.179656 20803 external_mini_cluster.cc:1428] Setting key ba88b77e8bbcfd21bd3ed5655e272bbf
I20260929 10:54:11.180061 9559 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:54:11.181905 9559 catalog_manager.cc:1347] Generated new cluster ID: cd17ec96084346bd86668aabc107e4ae
I20260929 10:54:11.181954 9559 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:54:11.186079 9568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:11.180230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:42995 (local address 127.0.0.1:46755)
0929 10:54:11.180524 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:11.180531 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:11.180555 (+ 24us) server_negotiation.cc:407] Connection header received
0929 10:54:11.180778 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:11.180788 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:11.180850 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:11.181250 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:11.181355 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:11.182722 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:11.183846 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:11.184443 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:11.184537 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:11.185308 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:11.185311 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:11.185312 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:54:11.185505 (+ 193us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:11.185544 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:54:11.185557 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:11.185637 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:11.185860 (+ 223us) server_negotiation.cc:299] Negotiation successful
0929 10:54:11.185887 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":67,"threads_started":1}
I20260929 10:54:11.187000 20803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:54:11.191850 9568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:11.188499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:35339 (local address 127.0.0.1:46755)
0929 10:54:11.188534 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:11.188536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:11.188543 (+ 7us) server_negotiation.cc:407] Connection header received
0929 10:54:11.188631 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:11.188632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:11.188642 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:11.188712 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:11.188848 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:11.189593 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:11.190551 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:11.190862 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:11.190936 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:11.191592 (+ 656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:11.191594 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:11.191594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:54:11.191598 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:11.191613 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:54:11.191617 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:11.191676 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:11.191755 (+ 79us) server_negotiation.cc:299] Negotiation successful
0929 10:54:11.191769 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 10:54:11.207115 9559 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:54:11.207827 9559 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:54:11.213011 9559 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 387409dfe6a149db91109fe52ea05723: Generated new TSK 0
I20260929 10:54:11.213189 9559 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:54:11.215049 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 9478
W20260929 10:54:11.219506 9462 connection.cc:535] client connection to 127.0.0.1:46755 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46755): Connection reset by peer (error 104)
2026-09-29T10:54:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1349 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:54:11 dist-test-slave-vqch krb5kdc[9578](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:54:11 dist-test-slave-vqch krb5kdc[9578](info): set up 2 sockets
Sep 29 10:54:11 dist-test-slave-vqch krb5kdc[9578](info): commencing operation
krb5kdc: starting...
W20260929 10:54:12.252771 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:54:12 dist-test-slave-vqch krb5kdc[9578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:54:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:54:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 10:54:12.367748 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37343
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37343
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:54:12.446887 9591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:54:12.447088 9591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:54:12.447117 9591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:54:12.448182 9591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:54:12.448215 9591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:54:12.448230 9591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:54:12.448241 9591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:54:12.448253 9591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:54:12.449695 9591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:54:12.449882 9591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:54:12.450040 9591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:54:12.454814 9599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:12.454811 9597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:12.454795 9596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:12.454995 9591 server_base.cc:1034] running on GCE node
I20260929 10:54:12.455353 9591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:54:12.455657 9591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:54:12.456815 9591 hybrid_clock.cc:648] HybridClock initialized: now 1790679252456802 us; error 29 us; skew 500 ppm
Sep 29 10:54:12 dist-test-slave-vqch krb5kdc[9578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679252, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:54:12.462888 9591 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 10:54:12.463326 9591 webserver.cc:460] Webserver started at http://127.0.0.1:35395/ using document root <none> and password file <none>
I20260929 10:54:12.463568 9591 fs_manager.cc:359] Metadata directory not provided
I20260929 10:54:12.463636 9591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:54:12.463786 9591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:54:12.464821 9591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "cb2d2fd448d044f980f989ee596b0c1e"
format_stamp: "Formatted at 2026-09-29 10:54:12 on dist-test-slave-vqch"
server_key: "0232afb8a66b4e4520e9cce97e767833"
server_key_iv: "53c6f1e73f50911946df16d37d972ba0"
server_key_version: "encryptionkey@0"
I20260929 10:54:12.465193 9591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "cb2d2fd448d044f980f989ee596b0c1e"
format_stamp: "Formatted at 2026-09-29 10:54:12 on dist-test-slave-vqch"
server_key: "0232afb8a66b4e4520e9cce97e767833"
server_key_iv: "53c6f1e73f50911946df16d37d972ba0"
server_key_version: "encryptionkey@0"
I20260929 10:54:12.466475 9591 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:54:12.467417 9606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:12.467612 9591 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:54:12.467720 9591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "cb2d2fd448d044f980f989ee596b0c1e"
format_stamp: "Formatted at 2026-09-29 10:54:12 on dist-test-slave-vqch"
server_key: "0232afb8a66b4e4520e9cce97e767833"
server_key_iv: "53c6f1e73f50911946df16d37d972ba0"
server_key_version: "encryptionkey@0"
I20260929 10:54:12.467801 9591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:54:12.471064 9591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:54:12.472946 9591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:54:12.473161 9591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:54:12.476819 9591 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37343
I20260929 10:54:12.476854 9658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37343 every 8 connection(s)
I20260929 10:54:12.477221 9591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:54:12.478084 9659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:54:12.480250 9659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb2d2fd448d044f980f989ee596b0c1e: Bootstrap starting.
I20260929 10:54:12.480815 9659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb2d2fd448d044f980f989ee596b0c1e: Neither blocks nor log segments found. Creating new log.
I20260929 10:54:12.481159 9659 log.cc:824] T 00000000000000000000000000000000 P cb2d2fd448d044f980f989ee596b0c1e: Log is configured to *not* fsync() on all Append() calls
I20260929 10:54:12.481869 9659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb2d2fd448d044f980f989ee596b0c1e: No bootstrap required, opened a new log
I20260929 10:54:12.481860 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 9591
I20260929 10:54:12.481994 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:54:12.482172 20803 external_mini_cluster.cc:1428] Setting key 281885928c41646f0ac3e6c3545c5219
I20260929 10:54:12.484120 9659 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb2d2fd448d044f980f989ee596b0c1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb2d2fd448d044f980f989ee596b0c1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37343 } }
I20260929 10:54:12.484354 9659 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb2d2fd448d044f980f989ee596b0c1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:54:12.484388 9659 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb2d2fd448d044f980f989ee596b0c1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb2d2fd448d044f980f989ee596b0c1e, State: Initialized, Role: FOLLOWER
I20260929 10:54:12.484512 9659 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb2d2fd448d044f980f989ee596b0c1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb2d2fd448d044f980f989ee596b0c1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37343 } }
I20260929 10:54:12.484596 9659 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb2d2fd448d044f980f989ee596b0c1e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:54:12.484689 9659 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb2d2fd448d044f980f989ee596b0c1e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:54:12.484747 9659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb2d2fd448d044f980f989ee596b0c1e [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:54:12.485422 9659 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb2d2fd448d044f980f989ee596b0c1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb2d2fd448d044f980f989ee596b0c1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37343 } }
I20260929 10:54:12.485524 9659 leader_election.cc:302] T 00000000000000000000000000000000 P cb2d2fd448d044f980f989ee596b0c1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb2d2fd448d044f980f989ee596b0c1e; no voters:
I20260929 10:54:12.485666 9663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:12.482703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:48041 (local address 127.0.0.1:37343)
0929 10:54:12.482897 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:12.482906 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:12.482938 (+ 32us) server_negotiation.cc:407] Connection header received
0929 10:54:12.484322 (+ 1384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:12.484330 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:12.484367 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:12.484518 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:12.485205 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:12.485209 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:12.485209 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:54:12.485214 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:12.485258 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:54:12.485273 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:12.485350 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:12.485404 (+ 54us) server_negotiation.cc:299] Negotiation successful
0929 10:54:12.485449 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":86,"threads_started":1}
I20260929 10:54:12.485683 9659 leader_election.cc:288] T 00000000000000000000000000000000 P cb2d2fd448d044f980f989ee596b0c1e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:54:12.485743 9664 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb2d2fd448d044f980f989ee596b0c1e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:54:12.485888 9664 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb2d2fd448d044f980f989ee596b0c1e [term 1 LEADER]: Becoming Leader. State: Replica: cb2d2fd448d044f980f989ee596b0c1e, State: Running, Role: LEADER
I20260929 10:54:12.485906 9659 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb2d2fd448d044f980f989ee596b0c1e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:54:12.486531 20803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:54:12.487349 9664 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb2d2fd448d044f980f989ee596b0c1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb2d2fd448d044f980f989ee596b0c1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37343 } }
I20260929 10:54:12.487879 9666 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb2d2fd448d044f980f989ee596b0c1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb2d2fd448d044f980f989ee596b0c1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb2d2fd448d044f980f989ee596b0c1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37343 } } }
I20260929 10:54:12.487979 9666 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb2d2fd448d044f980f989ee596b0c1e [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:12.488454 9682 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:54:12.488559 9665 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb2d2fd448d044f980f989ee596b0c1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb2d2fd448d044f980f989ee596b0c1e. Latest consensus state: current_term: 1 leader_uuid: "cb2d2fd448d044f980f989ee596b0c1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb2d2fd448d044f980f989ee596b0c1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37343 } } }
I20260929 10:54:12.488621 9665 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb2d2fd448d044f980f989ee596b0c1e [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:12.489173 9682 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:54:12.489316 9663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:12.488191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:55193 (local address 127.0.0.1:37343)
0929 10:54:12.488225 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:12.488227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:12.488283 (+ 56us) server_negotiation.cc:407] Connection header received
0929 10:54:12.488362 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:12.488363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:12.488367 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:12.488423 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:12.489019 (+ 596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:12.489020 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:12.489020 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:54:12.489025 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:12.489038 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:54:12.489041 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:12.489106 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:12.489241 (+ 135us) server_negotiation.cc:299] Negotiation successful
0929 10:54:12.489257 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 10:54:12.490671 9682 catalog_manager.cc:1347] Generated new cluster ID: ae0d0f78bb1a4123b1aae8eef956e826
I20260929 10:54:12.490723 9682 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:54:12.503957 9682 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:54:12.504127 9682 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:54:12.515890 9682 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb2d2fd448d044f980f989ee596b0c1e: Generated new TSK 0
I20260929 10:54:12.516086 9682 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:54:12.521667 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 9591
2026-09-29T10:54:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1306 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:54:12 dist-test-slave-vqch krb5kdc[9691](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:54:12 dist-test-slave-vqch krb5kdc[9691](info): set up 2 sockets
Sep 29 10:54:12 dist-test-slave-vqch krb5kdc[9691](info): commencing operation
krb5kdc: starting...
W20260929 10:54:13.548758 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:54:13 dist-test-slave-vqch krb5kdc[9691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:54:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:54:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 10:54:13.667369 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45129
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45129
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:54:13.747686 9704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:54:13.747927 9704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:54:13.747967 9704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:54:13.749181 9704 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:54:13.749243 9704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:54:13.749269 9704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:54:13.749305 9704 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:54:13.749330 9704 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:54:13.750803 9704 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45129
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:54:13.751070 9704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:54:13.751292 9704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:54:13.755992 9709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:13.756068 9712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:13.756250 9704 server_base.cc:1034] running on GCE node
W20260929 10:54:13.756036 9710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:13.756542 9704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:54:13.756824 9704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:54:13.757989 9704 hybrid_clock.cc:648] HybridClock initialized: now 1790679253758000 us; error 53 us; skew 500 ppm
Sep 29 10:54:13 dist-test-slave-vqch krb5kdc[9691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679253, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:54:13.760566 9704 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 10:54:13.761014 9704 webserver.cc:460] Webserver started at http://127.0.0.1:42873/ using document root <none> and password file <none>
I20260929 10:54:13.761240 9704 fs_manager.cc:359] Metadata directory not provided
I20260929 10:54:13.761308 9704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:54:13.761454 9704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:54:13.762526 9704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "fc3641ef841e478cba709cf691461131"
format_stamp: "Formatted at 2026-09-29 10:54:13 on dist-test-slave-vqch"
server_key: "da91fbf0c7716ea228477f35e0d730c5"
server_key_iv: "1d10c20a5881daa998637c1617ca550d"
server_key_version: "encryptionkey@0"
I20260929 10:54:13.762884 9704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "fc3641ef841e478cba709cf691461131"
format_stamp: "Formatted at 2026-09-29 10:54:13 on dist-test-slave-vqch"
server_key: "da91fbf0c7716ea228477f35e0d730c5"
server_key_iv: "1d10c20a5881daa998637c1617ca550d"
server_key_version: "encryptionkey@0"
I20260929 10:54:13.764176 9704 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:54:13.764865 9719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:13.765069 9704 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:54:13.765165 9704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "fc3641ef841e478cba709cf691461131"
format_stamp: "Formatted at 2026-09-29 10:54:13 on dist-test-slave-vqch"
server_key: "da91fbf0c7716ea228477f35e0d730c5"
server_key_iv: "1d10c20a5881daa998637c1617ca550d"
server_key_version: "encryptionkey@0"
I20260929 10:54:13.765249 9704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:54:13.783926 9704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:54:13.785903 9704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:54:13.786063 9704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:54:13.790141 9704 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45129
I20260929 10:54:13.790190 9771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45129 every 8 connection(s)
I20260929 10:54:13.790563 9704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:54:13.791426 9772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:54:13.792062 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 9704
I20260929 10:54:13.792181 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:54:13.792374 20803 external_mini_cluster.cc:1428] Setting key f0bbd1daed5b4488026d551fcafd1aef
I20260929 10:54:13.794567 9772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc3641ef841e478cba709cf691461131: Bootstrap starting.
I20260929 10:54:13.795187 9772 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc3641ef841e478cba709cf691461131: Neither blocks nor log segments found. Creating new log.
I20260929 10:54:13.795482 9772 log.cc:824] T 00000000000000000000000000000000 P fc3641ef841e478cba709cf691461131: Log is configured to *not* fsync() on all Append() calls
I20260929 10:54:13.796141 9772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc3641ef841e478cba709cf691461131: No bootstrap required, opened a new log
I20260929 10:54:13.798215 9772 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc3641ef841e478cba709cf691461131 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3641ef841e478cba709cf691461131" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45129 } }
I20260929 10:54:13.798380 9772 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc3641ef841e478cba709cf691461131 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:54:13.798422 9772 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc3641ef841e478cba709cf691461131 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc3641ef841e478cba709cf691461131, State: Initialized, Role: FOLLOWER
I20260929 10:54:13.798554 9772 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc3641ef841e478cba709cf691461131 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3641ef841e478cba709cf691461131" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45129 } }
I20260929 10:54:13.798629 9772 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc3641ef841e478cba709cf691461131 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:54:13.798657 9772 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc3641ef841e478cba709cf691461131 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:54:13.798696 9772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc3641ef841e478cba709cf691461131 [term 0 FOLLOWER]: Advancing to term 1
Sep 29 10:54:13 dist-test-slave-vqch krb5kdc[9691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 10:54:13.799369 9772 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc3641ef841e478cba709cf691461131 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3641ef841e478cba709cf691461131" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45129 } }
I20260929 10:54:13.799494 9772 leader_election.cc:302] T 00000000000000000000000000000000 P fc3641ef841e478cba709cf691461131 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc3641ef841e478cba709cf691461131; no voters:
I20260929 10:54:13.799667 9772 leader_election.cc:288] T 00000000000000000000000000000000 P fc3641ef841e478cba709cf691461131 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:54:13.799695 9777 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc3641ef841e478cba709cf691461131 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:54:13.799817 9777 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc3641ef841e478cba709cf691461131 [term 1 LEADER]: Becoming Leader. State: Replica: fc3641ef841e478cba709cf691461131, State: Running, Role: LEADER
I20260929 10:54:13.799953 9772 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc3641ef841e478cba709cf691461131 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:54:13.799980 9777 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc3641ef841e478cba709cf691461131 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3641ef841e478cba709cf691461131" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45129 } }
I20260929 10:54:13.800385 9778 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc3641ef841e478cba709cf691461131 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc3641ef841e478cba709cf691461131" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3641ef841e478cba709cf691461131" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45129 } } }
I20260929 10:54:13.800518 9778 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc3641ef841e478cba709cf691461131 [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:13.800750 9779 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc3641ef841e478cba709cf691461131 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc3641ef841e478cba709cf691461131. Latest consensus state: current_term: 1 leader_uuid: "fc3641ef841e478cba709cf691461131" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3641ef841e478cba709cf691461131" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45129 } } }
I20260929 10:54:13.800904 9779 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc3641ef841e478cba709cf691461131 [sys.catalog]: This master's current role is: LEADER
W20260929 10:54:13.801800 9793 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fc3641ef841e478cba709cf691461131: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:54:13.801854 9793 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:54:13.801901 9786 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:54:13.802058 9786 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:54:13.803663 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:13.793023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:44367 (local address 127.0.0.1:45129)
0929 10:54:13.793195 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:13.793204 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:13.793238 (+ 34us) server_negotiation.cc:407] Connection header received
0929 10:54:13.793652 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:13.793658 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:13.793706 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:13.793821 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:13.794604 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:13.795627 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:13.796445 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:13.796859 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:13.796970 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:13.799678 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:13.799682 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:13.799683 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:13.799693 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:13.801925 (+ 2232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:13.802399 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:13.802402 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:13.802403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:13.802492 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:13.802744 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:13.802745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:13.802745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:13.803061 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:13.803234 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:13.803366 (+ 132us) server_negotiation.cc:299] Negotiation successful
0929 10:54:13.803469 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":104,"threads_started":1}
I20260929 10:54:13.804751 20803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:54:13.805363 9786 catalog_manager.cc:1347] Generated new cluster ID: 1d9aa299c45d496d8a9a3f4ad74d5fd9
I20260929 10:54:13.805418 9786 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:54:13.813663 9786 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:54:13.814057 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:13.806299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:52419 (local address 127.0.0.1:45129)
0929 10:54:13.806332 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:13.806334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:13.806434 (+ 100us) server_negotiation.cc:407] Connection header received
0929 10:54:13.806494 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:13.806494 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:13.806503 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:13.806556 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:13.807186 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:13.808178 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:13.809223 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:13.809558 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:13.809623 (+ 65us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:13.811105 (+ 1482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:13.811106 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:13.811106 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:13.811118 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:13.812721 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:13.813193 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:13.813194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:13.813195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:13.813255 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:13.813491 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:13.813492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:13.813492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:13.813650 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:13.813787 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:13.813890 (+ 103us) server_negotiation.cc:299] Negotiation successful
0929 10:54:13.813957 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 10:54:13.814508 9786 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:54:13.831315 9786 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc3641ef841e478cba709cf691461131: Generated new TSK 0
I20260929 10:54:13.831550 9786 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:54:13.835412 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 9704
W20260929 10:54:13.839905 9689 connection.cc:535] client connection to 127.0.0.1:45129 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45129): Connection reset by peer (error 104)
2026-09-29T10:54:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1313 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:54:13 dist-test-slave-vqch krb5kdc[9803](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:54:13 dist-test-slave-vqch krb5kdc[9803](info): set up 2 sockets
Sep 29 10:54:13 dist-test-slave-vqch krb5kdc[9803](info): commencing operation
krb5kdc: starting...
W20260929 10:54:14.876859 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:54:14 dist-test-slave-vqch krb5kdc[9803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:54:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:54:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 10:54:14.996124 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37201
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:54:15.076009 9816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:54:15.076236 9816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:54:15.076269 9816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:54:15.077448 9816 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:54:15.077505 9816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:54:15.077529 9816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:54:15.077561 9816 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:54:15.077589 9816 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:54:15.079042 9816 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37201
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:54:15.079308 9816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:54:15.079522 9816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:54:15.084126 9824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:15.084173 9821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:15.084296 9816 server_base.cc:1034] running on GCE node
W20260929 10:54:15.084146 9822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:15.084606 9816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:54:15.084879 9816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:54:15.086020 9816 hybrid_clock.cc:648] HybridClock initialized: now 1790679255086024 us; error 45 us; skew 500 ppm
Sep 29 10:54:15 dist-test-slave-vqch krb5kdc[9803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679255, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:54:15.088317 9816 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 10:54:15.088697 9816 webserver.cc:460] Webserver started at http://127.0.0.1:46173/ using document root <none> and password file <none>
I20260929 10:54:15.088896 9816 fs_manager.cc:359] Metadata directory not provided
I20260929 10:54:15.088936 9816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:54:15.089082 9816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:54:15.090127 9816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "852873df14d24299a608b4fc7f452360"
format_stamp: "Formatted at 2026-09-29 10:54:15 on dist-test-slave-vqch"
server_key: "bc4585f83ea21f30ed30a25cd667cea4"
server_key_iv: "c399f96d8ac1eef9900ca1a8ff96a31f"
server_key_version: "encryptionkey@0"
I20260929 10:54:15.090425 9816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "852873df14d24299a608b4fc7f452360"
format_stamp: "Formatted at 2026-09-29 10:54:15 on dist-test-slave-vqch"
server_key: "bc4585f83ea21f30ed30a25cd667cea4"
server_key_iv: "c399f96d8ac1eef9900ca1a8ff96a31f"
server_key_version: "encryptionkey@0"
I20260929 10:54:15.091645 9816 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:54:15.092316 9832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:15.092497 9816 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:54:15.092567 9816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "852873df14d24299a608b4fc7f452360"
format_stamp: "Formatted at 2026-09-29 10:54:15 on dist-test-slave-vqch"
server_key: "bc4585f83ea21f30ed30a25cd667cea4"
server_key_iv: "c399f96d8ac1eef9900ca1a8ff96a31f"
server_key_version: "encryptionkey@0"
I20260929 10:54:15.092620 9816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:54:15.104858 9816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:54:15.106695 9816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:54:15.106810 9816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:54:15.110713 9816 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37201
I20260929 10:54:15.110801 9884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37201 every 8 connection(s)
I20260929 10:54:15.111114 9816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:54:15.111981 9885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:54:15.114326 9885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 852873df14d24299a608b4fc7f452360: Bootstrap starting.
I20260929 10:54:15.114902 9885 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 852873df14d24299a608b4fc7f452360: Neither blocks nor log segments found. Creating new log.
I20260929 10:54:15.115183 9885 log.cc:824] T 00000000000000000000000000000000 P 852873df14d24299a608b4fc7f452360: Log is configured to *not* fsync() on all Append() calls
I20260929 10:54:15.115864 9885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 852873df14d24299a608b4fc7f452360: No bootstrap required, opened a new log
I20260929 10:54:15.117506 9885 raft_consensus.cc:348] T 00000000000000000000000000000000 P 852873df14d24299a608b4fc7f452360 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "852873df14d24299a608b4fc7f452360" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37201 } }
I20260929 10:54:15.117683 9885 raft_consensus.cc:374] T 00000000000000000000000000000000 P 852873df14d24299a608b4fc7f452360 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:54:15.117729 9885 raft_consensus.cc:729] T 00000000000000000000000000000000 P 852873df14d24299a608b4fc7f452360 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 852873df14d24299a608b4fc7f452360, State: Initialized, Role: FOLLOWER
I20260929 10:54:15.117888 9885 consensus_queue.cc:260] T 00000000000000000000000000000000 P 852873df14d24299a608b4fc7f452360 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "852873df14d24299a608b4fc7f452360" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37201 } }
I20260929 10:54:15.117967 9885 raft_consensus.cc:388] T 00000000000000000000000000000000 P 852873df14d24299a608b4fc7f452360 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:54:15.118012 9885 raft_consensus.cc:482] T 00000000000000000000000000000000 P 852873df14d24299a608b4fc7f452360 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:54:15.118064 9885 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 852873df14d24299a608b4fc7f452360 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:54:15.118577 9885 raft_consensus.cc:504] T 00000000000000000000000000000000 P 852873df14d24299a608b4fc7f452360 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "852873df14d24299a608b4fc7f452360" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37201 } }
I20260929 10:54:15.118692 9885 leader_election.cc:302] T 00000000000000000000000000000000 P 852873df14d24299a608b4fc7f452360 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 852873df14d24299a608b4fc7f452360; no voters:
I20260929 10:54:15.118870 9885 leader_election.cc:288] T 00000000000000000000000000000000 P 852873df14d24299a608b4fc7f452360 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:54:15.118896 9888 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 852873df14d24299a608b4fc7f452360 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:54:15.119048 9888 raft_consensus.cc:686] T 00000000000000000000000000000000 P 852873df14d24299a608b4fc7f452360 [term 1 LEADER]: Becoming Leader. State: Replica: 852873df14d24299a608b4fc7f452360, State: Running, Role: LEADER
I20260929 10:54:15.119117 9885 sys_catalog.cc:564] T 00000000000000000000000000000000 P 852873df14d24299a608b4fc7f452360 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:54:15.119184 9888 consensus_queue.cc:237] T 00000000000000000000000000000000 P 852873df14d24299a608b4fc7f452360 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "852873df14d24299a608b4fc7f452360" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37201 } }
I20260929 10:54:15.119531 9890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 852873df14d24299a608b4fc7f452360 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 852873df14d24299a608b4fc7f452360. Latest consensus state: current_term: 1 leader_uuid: "852873df14d24299a608b4fc7f452360" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "852873df14d24299a608b4fc7f452360" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37201 } } }
I20260929 10:54:15.119663 9889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 852873df14d24299a608b4fc7f452360 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "852873df14d24299a608b4fc7f452360" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "852873df14d24299a608b4fc7f452360" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37201 } } }
I20260929 10:54:15.119679 9890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 852873df14d24299a608b4fc7f452360 [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:15.119757 9889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 852873df14d24299a608b4fc7f452360 [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:15.120122 9897 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:54:15.120659 9897 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:54:15.120687 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 9816
I20260929 10:54:15.120813 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:54:15.121078 20803 external_mini_cluster.cc:1428] Setting key 966fafd21488351ac71a8876fc4de48e
I20260929 10:54:15.122381 9897 catalog_manager.cc:1347] Generated new cluster ID: b761eaa4e3f34ed88230557a2eefbde5
I20260929 10:54:15.122438 9897 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:54:15.127151 9897 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:54:15.127313 9907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:15.121642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:50995 (local address 127.0.0.1:37201)
0929 10:54:15.121866 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:15.121875 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:15.121908 (+ 33us) server_negotiation.cc:407] Connection header received
0929 10:54:15.122027 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:15.122038 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:15.122083 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:15.122217 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:15.122432 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:15.123817 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:15.124940 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:15.125575 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:15.125674 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:15.126469 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:15.126473 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:15.126474 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:54:15.126682 (+ 208us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:15.126721 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:54:15.126735 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:15.126845 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:15.127025 (+ 180us) server_negotiation.cc:299] Negotiation successful
0929 10:54:15.127057 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":84,"threads_started":1}
I20260929 10:54:15.127821 9897 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:54:15.128227 20803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:54:15.131814 9897 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 852873df14d24299a608b4fc7f452360: Generated new TSK 0
I20260929 10:54:15.131974 9897 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:54:15.134506 9907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:15.129665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:36287 (local address 127.0.0.1:37201)
0929 10:54:15.129695 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:15.129696 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:15.131591 (+ 1895us) server_negotiation.cc:407] Connection header received
0929 10:54:15.131663 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:15.131664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:15.131677 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:15.131734 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:15.131872 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:15.132458 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:15.133286 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:15.133521 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:15.133603 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:15.134282 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:15.134283 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:15.134283 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:54:15.134288 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:15.134301 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:54:15.134306 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:15.134357 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:15.134419 (+ 62us) server_negotiation.cc:299] Negotiation successful
0929 10:54:15.134430 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260929 10:54:15.136178 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 9816
W20260929 10:54:15.140714 9801 connection.cc:535] client connection to 127.0.0.1:37201 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37201): Connection reset by peer (error 104)
2026-09-29T10:54:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1300 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:54:15 dist-test-slave-vqch krb5kdc[9916](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:54:15 dist-test-slave-vqch krb5kdc[9916](info): set up 2 sockets
Sep 29 10:54:15 dist-test-slave-vqch krb5kdc[9916](info): commencing operation
krb5kdc: starting...
W20260929 10:54:16.164824 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:54:16 dist-test-slave-vqch krb5kdc[9916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:54:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:54:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 10:54:16.280690 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46683
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:54:16.360494 9929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:54:16.360689 9929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:54:16.360711 9929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:54:16.361972 9929 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:54:16.362013 9929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:54:16.362027 9929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:54:16.362040 9929 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:54:16.362084 9929 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:54:16.363517 9929 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46683
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790678996126930-20803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:54:16.363744 9929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:54:16.363979 9929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:54:16.368736 9937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:16.368831 9934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:16.369055 9929 server_base.cc:1034] running on GCE node
W20260929 10:54:16.368817 9935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:16.369361 9929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:54:16.369637 9929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:54:16.370792 9929 hybrid_clock.cc:648] HybridClock initialized: now 1790679256370773 us; error 36 us; skew 500 ppm
Sep 29 10:54:16 dist-test-slave-vqch krb5kdc[9916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679256, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:54:16.373193 9929 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 10:54:16.373616 9929 webserver.cc:460] Webserver started at http://127.0.0.1:33683/ using document root <none> and password file <none>
I20260929 10:54:16.373858 9929 fs_manager.cc:359] Metadata directory not provided
I20260929 10:54:16.373924 9929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:54:16.374058 9929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:54:16.375118 9929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "c492544c2e2d4094b8070a690d7870c4"
format_stamp: "Formatted at 2026-09-29 10:54:16 on dist-test-slave-vqch"
server_key: "d3c2d128210f99d13ad88f033b38b904"
server_key_iv: "a03e7ed2f7ddd70c68b03cd76e3f256e"
server_key_version: "encryptionkey@0"
I20260929 10:54:16.375471 9929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "c492544c2e2d4094b8070a690d7870c4"
format_stamp: "Formatted at 2026-09-29 10:54:16 on dist-test-slave-vqch"
server_key: "d3c2d128210f99d13ad88f033b38b904"
server_key_iv: "a03e7ed2f7ddd70c68b03cd76e3f256e"
server_key_version: "encryptionkey@0"
I20260929 10:54:16.376811 9929 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260929 10:54:16.377635 9944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:16.377808 9929 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 10:54:16.377904 9929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "c492544c2e2d4094b8070a690d7870c4"
format_stamp: "Formatted at 2026-09-29 10:54:16 on dist-test-slave-vqch"
server_key: "d3c2d128210f99d13ad88f033b38b904"
server_key_iv: "a03e7ed2f7ddd70c68b03cd76e3f256e"
server_key_version: "encryptionkey@0"
I20260929 10:54:16.377982 9929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:54:16.381256 9929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:54:16.383160 9929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:54:16.383320 9929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:54:16.387236 9929 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46683
I20260929 10:54:16.387269 9997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46683 every 8 connection(s)
I20260929 10:54:16.387612 9929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:54:16.388432 9998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:54:16.390729 9998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c492544c2e2d4094b8070a690d7870c4: Bootstrap starting.
I20260929 10:54:16.391304 9998 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c492544c2e2d4094b8070a690d7870c4: Neither blocks nor log segments found. Creating new log.
I20260929 10:54:16.391580 9998 log.cc:824] T 00000000000000000000000000000000 P c492544c2e2d4094b8070a690d7870c4: Log is configured to *not* fsync() on all Append() calls
I20260929 10:54:16.392246 9998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c492544c2e2d4094b8070a690d7870c4: No bootstrap required, opened a new log
I20260929 10:54:16.393883 9998 raft_consensus.cc:348] T 00000000000000000000000000000000 P c492544c2e2d4094b8070a690d7870c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c492544c2e2d4094b8070a690d7870c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46683 } }
I20260929 10:54:16.394019 9998 raft_consensus.cc:374] T 00000000000000000000000000000000 P c492544c2e2d4094b8070a690d7870c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:54:16.394071 9998 raft_consensus.cc:729] T 00000000000000000000000000000000 P c492544c2e2d4094b8070a690d7870c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c492544c2e2d4094b8070a690d7870c4, State: Initialized, Role: FOLLOWER
I20260929 10:54:16.394280 9998 consensus_queue.cc:260] T 00000000000000000000000000000000 P c492544c2e2d4094b8070a690d7870c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c492544c2e2d4094b8070a690d7870c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46683 } }
I20260929 10:54:16.394361 9998 raft_consensus.cc:388] T 00000000000000000000000000000000 P c492544c2e2d4094b8070a690d7870c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:54:16.394408 9998 raft_consensus.cc:482] T 00000000000000000000000000000000 P c492544c2e2d4094b8070a690d7870c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:54:16.394459 9998 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c492544c2e2d4094b8070a690d7870c4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:54:16.394990 9998 raft_consensus.cc:504] T 00000000000000000000000000000000 P c492544c2e2d4094b8070a690d7870c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c492544c2e2d4094b8070a690d7870c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46683 } }
I20260929 10:54:16.395107 9998 leader_election.cc:302] T 00000000000000000000000000000000 P c492544c2e2d4094b8070a690d7870c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c492544c2e2d4094b8070a690d7870c4; no voters:
I20260929 10:54:16.395109 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 9929
I20260929 10:54:16.395237 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:54:16.395310 9998 leader_election.cc:288] T 00000000000000000000000000000000 P c492544c2e2d4094b8070a690d7870c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:54:16.395357 10001 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c492544c2e2d4094b8070a690d7870c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:54:16.395455 20803 external_mini_cluster.cc:1428] Setting key f9e8fb020b25b3fb10f2a5291112932e
I20260929 10:54:16.395517 10001 raft_consensus.cc:686] T 00000000000000000000000000000000 P c492544c2e2d4094b8070a690d7870c4 [term 1 LEADER]: Becoming Leader. State: Replica: c492544c2e2d4094b8070a690d7870c4, State: Running, Role: LEADER
I20260929 10:54:16.395562 9998 sys_catalog.cc:564] T 00000000000000000000000000000000 P c492544c2e2d4094b8070a690d7870c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:54:16.395601 10001 consensus_queue.cc:237] T 00000000000000000000000000000000 P c492544c2e2d4094b8070a690d7870c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c492544c2e2d4094b8070a690d7870c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46683 } }
I20260929 10:54:16.396214 10003 sys_catalog.cc:455] T 00000000000000000000000000000000 P c492544c2e2d4094b8070a690d7870c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c492544c2e2d4094b8070a690d7870c4. Latest consensus state: current_term: 1 leader_uuid: "c492544c2e2d4094b8070a690d7870c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c492544c2e2d4094b8070a690d7870c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46683 } } }
I20260929 10:54:16.396313 10003 sys_catalog.cc:458] T 00000000000000000000000000000000 P c492544c2e2d4094b8070a690d7870c4 [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:16.396551 10002 sys_catalog.cc:455] T 00000000000000000000000000000000 P c492544c2e2d4094b8070a690d7870c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c492544c2e2d4094b8070a690d7870c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c492544c2e2d4094b8070a690d7870c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46683 } } }
I20260929 10:54:16.396662 10002 sys_catalog.cc:458] T 00000000000000000000000000000000 P c492544c2e2d4094b8070a690d7870c4 [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:16.396572 10011 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:54:16.397326 10011 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:54:16.398478 10006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:16.396086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:51249 (local address 127.0.0.1:46683)
0929 10:54:16.396748 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:16.396757 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:16.396793 (+ 36us) server_negotiation.cc:407] Connection header received
0929 10:54:16.396907 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:16.396914 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:16.396946 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:16.397475 (+ 529us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:16.398006 (+ 531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:16.398009 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:16.398009 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:54:16.398013 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:16.398047 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:54:16.398060 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:16.398144 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:16.398218 (+ 74us) server_negotiation.cc:299] Negotiation successful
0929 10:54:16.398263 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":71,"threads_started":1}
I20260929 10:54:16.399060 10011 catalog_manager.cc:1347] Generated new cluster ID: 6dae8577449a40e5a2f417a7adfb55ec
I20260929 10:54:16.399113 10011 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:54:16.399346 20803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:54:16.410218 10006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:16.401246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:45625 (local address 127.0.0.1:46683)
0929 10:54:16.401294 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:16.401297 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:16.409154 (+ 7857us) server_negotiation.cc:407] Connection header received
0929 10:54:16.409216 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:16.409217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:16.409223 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:16.409288 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:16.409936 (+ 648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:16.409937 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:16.409938 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:54:16.409942 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:16.409961 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:54:16.409966 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:16.410055 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:16.410110 (+ 55us) server_negotiation.cc:299] Negotiation successful
0929 10:54:16.410124 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 10:54:16.423985 10011 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:54:16.424187 10011 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:54:16.429251 10011 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c492544c2e2d4094b8070a690d7870c4: Generated new TSK 0
I20260929 10:54:16.429452 10011 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:54:16.431416 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 9929
2026-09-29T10:54:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1295 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:54:16 dist-test-slave-vqch krb5kdc[10029](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:54:16 dist-test-slave-vqch krb5kdc[10029](info): set up 2 sockets
Sep 29 10:54:16 dist-test-slave-vqch krb5kdc[10029](info): commencing operation
krb5kdc: starting...
W20260929 10:54:17.496661 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:54:17 dist-test-slave-vqch krb5kdc[10029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:54:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:54:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.80.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
I20260929 10:54:17.617080 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:42677
--webserver_interface=127.20.80.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:38065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:42677
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:54:17.700498 10042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:54:17.700717 10042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:54:17.700742 10042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:54:17.701968 10042 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:54:17.702013 10042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:54:17.702028 10042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:54:17.702040 10042 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:54:17.702052 10042 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:54:17.703572 10042 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:38065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:42677
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:42677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:54:17.703819 10042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:54:17.704046 10042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 10:54:17.709040 10042 server_base.cc:1034] running on GCE node
W20260929 10:54:17.708997 10047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:17.709015 10048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:17.709020 10050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:17.709438 10042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:54:17.709676 10042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:54:17.710824 10042 hybrid_clock.cc:648] HybridClock initialized: now 1790679257710804 us; error 38 us; skew 500 ppm
Sep 29 10:54:17 dist-test-slave-vqch krb5kdc[10029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679257, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:54:17.718981 10042 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:54:17.719463 10042 webserver.cc:460] Webserver started at http://127.20.80.254:39087/ using document root <none> and password file <none>
I20260929 10:54:17.719707 10042 fs_manager.cc:359] Metadata directory not provided
I20260929 10:54:17.719753 10042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:54:17.719918 10042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:54:17.721050 10042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "325d73e168c34ff1b58917ca2260a292"
format_stamp: "Formatted at 2026-09-29 10:54:17 on dist-test-slave-vqch"
server_key: "dbab00df32c89d58efb26c98f9f2cd2c"
server_key_iv: "75766b29e47612eb21cfcca114bdd795"
server_key_version: "encryptionkey@0"
I20260929 10:54:17.721493 10042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "325d73e168c34ff1b58917ca2260a292"
format_stamp: "Formatted at 2026-09-29 10:54:17 on dist-test-slave-vqch"
server_key: "dbab00df32c89d58efb26c98f9f2cd2c"
server_key_iv: "75766b29e47612eb21cfcca114bdd795"
server_key_version: "encryptionkey@0"
I20260929 10:54:17.722811 10042 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 10:54:17.723547 10057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:17.723718 10042 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:54:17.723816 10042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "325d73e168c34ff1b58917ca2260a292"
format_stamp: "Formatted at 2026-09-29 10:54:17 on dist-test-slave-vqch"
server_key: "dbab00df32c89d58efb26c98f9f2cd2c"
server_key_iv: "75766b29e47612eb21cfcca114bdd795"
server_key_version: "encryptionkey@0"
I20260929 10:54:17.723886 10042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:54:17.734385 10042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:54:17.736157 10042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:54:17.736307 10042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:54:17.740787 10042 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:42677
I20260929 10:54:17.740845 10109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:42677 every 8 connection(s)
I20260929 10:54:17.741221 10042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:54:17.741711 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 10042
I20260929 10:54:17.741828 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:54:17.742015 20803 external_mini_cluster.cc:1428] Setting key f1812af518e2b772c59846b2d3d8e706
I20260929 10:54:17.742153 10110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:54:17.745476 10110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 325d73e168c34ff1b58917ca2260a292: Bootstrap starting.
I20260929 10:54:17.746160 10110 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 325d73e168c34ff1b58917ca2260a292: Neither blocks nor log segments found. Creating new log.
I20260929 10:54:17.746464 10110 log.cc:824] T 00000000000000000000000000000000 P 325d73e168c34ff1b58917ca2260a292: Log is configured to *not* fsync() on all Append() calls
I20260929 10:54:17.747153 10110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 325d73e168c34ff1b58917ca2260a292: No bootstrap required, opened a new log
Sep 29 10:54:17 dist-test-slave-vqch krb5kdc[10029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:54:17.749419 10110 raft_consensus.cc:348] T 00000000000000000000000000000000 P 325d73e168c34ff1b58917ca2260a292 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325d73e168c34ff1b58917ca2260a292" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 42677 } }
I20260929 10:54:17.749609 10110 raft_consensus.cc:374] T 00000000000000000000000000000000 P 325d73e168c34ff1b58917ca2260a292 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:54:17.749689 10110 raft_consensus.cc:729] T 00000000000000000000000000000000 P 325d73e168c34ff1b58917ca2260a292 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 325d73e168c34ff1b58917ca2260a292, State: Initialized, Role: FOLLOWER
I20260929 10:54:17.749828 10110 consensus_queue.cc:260] T 00000000000000000000000000000000 P 325d73e168c34ff1b58917ca2260a292 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325d73e168c34ff1b58917ca2260a292" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 42677 } }
I20260929 10:54:17.749917 10110 raft_consensus.cc:388] T 00000000000000000000000000000000 P 325d73e168c34ff1b58917ca2260a292 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:54:17.749949 10110 raft_consensus.cc:482] T 00000000000000000000000000000000 P 325d73e168c34ff1b58917ca2260a292 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:54:17.749997 10110 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 325d73e168c34ff1b58917ca2260a292 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:54:17.750591 10110 raft_consensus.cc:504] T 00000000000000000000000000000000 P 325d73e168c34ff1b58917ca2260a292 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325d73e168c34ff1b58917ca2260a292" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 42677 } }
I20260929 10:54:17.750726 10110 leader_election.cc:302] T 00000000000000000000000000000000 P 325d73e168c34ff1b58917ca2260a292 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 325d73e168c34ff1b58917ca2260a292; no voters:
I20260929 10:54:17.750886 10110 leader_election.cc:288] T 00000000000000000000000000000000 P 325d73e168c34ff1b58917ca2260a292 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:54:17.750917 10115 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 325d73e168c34ff1b58917ca2260a292 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:54:17.751021 10115 raft_consensus.cc:686] T 00000000000000000000000000000000 P 325d73e168c34ff1b58917ca2260a292 [term 1 LEADER]: Becoming Leader. State: Replica: 325d73e168c34ff1b58917ca2260a292, State: Running, Role: LEADER
I20260929 10:54:17.751142 10110 sys_catalog.cc:564] T 00000000000000000000000000000000 P 325d73e168c34ff1b58917ca2260a292 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:54:17.751149 10115 consensus_queue.cc:237] T 00000000000000000000000000000000 P 325d73e168c34ff1b58917ca2260a292 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325d73e168c34ff1b58917ca2260a292" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 42677 } }
I20260929 10:54:17.751470 10115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 325d73e168c34ff1b58917ca2260a292 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 325d73e168c34ff1b58917ca2260a292. Latest consensus state: current_term: 1 leader_uuid: "325d73e168c34ff1b58917ca2260a292" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325d73e168c34ff1b58917ca2260a292" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 42677 } } }
I20260929 10:54:17.751564 10115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 325d73e168c34ff1b58917ca2260a292 [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:17.751480 10117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 325d73e168c34ff1b58917ca2260a292 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "325d73e168c34ff1b58917ca2260a292" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325d73e168c34ff1b58917ca2260a292" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 42677 } } }
I20260929 10:54:17.751722 10117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 325d73e168c34ff1b58917ca2260a292 [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:17.751787 10121 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:54:17.752311 10121 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:54:17.753856 10121 catalog_manager.cc:1347] Generated new cluster ID: 682f12d6a6a542b38d303e31c2a8afe9
I20260929 10:54:17.753928 10121 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:54:17.754069 10113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:17.742554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50951 (local address 127.20.80.254:42677)
0929 10:54:17.742736 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:17.742745 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:17.742779 (+ 34us) server_negotiation.cc:407] Connection header received
0929 10:54:17.743315 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:17.743321 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:17.743369 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:17.743487 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:17.744289 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:17.745253 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:17.746331 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:17.746792 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:17.746916 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:17.749576 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:17.749580 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:17.749580 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:17.749591 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:17.751623 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:17.752098 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:17.752101 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:17.752102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:17.752209 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:17.752495 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:17.752495 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:17.752496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:17.752920 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:17.753584 (+ 664us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:17.753668 (+ 84us) server_negotiation.cc:299] Negotiation successful
0929 10:54:17.753779 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":99,"threads_started":1}
I20260929 10:54:17.755005 20803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:54:17.764248 10113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:17.757067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53373 (local address 127.20.80.254:42677)
0929 10:54:17.757115 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:17.757117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:17.757125 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:54:17.757145 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:17.757146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:17.757155 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:17.757218 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:17.757802 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:17.758844 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:17.759790 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:17.760143 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:17.760218 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:17.761515 (+ 1297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:17.761516 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:17.761517 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:17.761523 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:17.763097 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:17.763488 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:17.763489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:17.763489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:17.763548 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:17.763765 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:17.763766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:17.763766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:17.763892 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:17.763995 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:17.764103 (+ 108us) server_negotiation.cc:299] Negotiation successful
0929 10:54:17.764164 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260929 10:54:17.770288 10121 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:54:17.770773 10121 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:54:17.777639 10121 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 325d73e168c34ff1b58917ca2260a292: Generated new TSK 0
I20260929 10:54:17.777809 10121 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:54:17.782500 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 10042
W20260929 10:54:17.787086 10027 connection.cc:535] client connection to 127.20.80.254:42677 recv error: Network error: failed to read from TLS socket (remote: 127.20.80.254:42677): Connection reset by peer (error 104)
2026-09-29T10:54:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1350 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:54:17 dist-test-slave-vqch krb5kdc[10141](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:54:17 dist-test-slave-vqch krb5kdc[10141](info): set up 2 sockets
Sep 29 10:54:17 dist-test-slave-vqch krb5kdc[10141](info): commencing operation
krb5kdc: starting...
W20260929 10:54:18.837116 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:54:18 dist-test-slave-vqch krb5kdc[10141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:54:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:54:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.80.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
I20260929 10:54:18.959982 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:44521
--webserver_interface=127.20.80.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:35671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:44521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:54:19.039247 10154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:54:19.039440 10154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:54:19.039462 10154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:54:19.040588 10154 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:54:19.040630 10154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:54:19.040644 10154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:54:19.040657 10154 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:54:19.040704 10154 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:54:19.042133 10154 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:35671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:44521
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:44521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:54:19.042369 10154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:54:19.042589 10154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:54:19.047271 10160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:19.047307 10162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:19.047436 10154 server_base.cc:1034] running on GCE node
W20260929 10:54:19.047333 10159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:19.047734 10154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:54:19.047972 10154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:54:19.049109 10154 hybrid_clock.cc:648] HybridClock initialized: now 1790679259049088 us; error 34 us; skew 500 ppm
Sep 29 10:54:19 dist-test-slave-vqch krb5kdc[10141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679259, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:54:19.051427 10154 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:54:19.051848 10154 webserver.cc:460] Webserver started at http://127.20.80.254:42195/ using document root <none> and password file <none>
I20260929 10:54:19.052081 10154 fs_manager.cc:359] Metadata directory not provided
I20260929 10:54:19.052146 10154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:54:19.052304 10154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:54:19.053406 10154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "f794235d908a4b179f7d0fcd8ee4f39f"
format_stamp: "Formatted at 2026-09-29 10:54:19 on dist-test-slave-vqch"
server_key: "f93147d68c11f3c5210e739cc9d142bc"
server_key_iv: "cb32305d48436c50e5afb9d5a40953f8"
server_key_version: "encryptionkey@0"
I20260929 10:54:19.053762 10154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "f794235d908a4b179f7d0fcd8ee4f39f"
format_stamp: "Formatted at 2026-09-29 10:54:19 on dist-test-slave-vqch"
server_key: "f93147d68c11f3c5210e739cc9d142bc"
server_key_iv: "cb32305d48436c50e5afb9d5a40953f8"
server_key_version: "encryptionkey@0"
I20260929 10:54:19.055052 10154 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:54:19.055707 10169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:19.055867 10154 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:54:19.055966 10154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "f794235d908a4b179f7d0fcd8ee4f39f"
format_stamp: "Formatted at 2026-09-29 10:54:19 on dist-test-slave-vqch"
server_key: "f93147d68c11f3c5210e739cc9d142bc"
server_key_iv: "cb32305d48436c50e5afb9d5a40953f8"
server_key_version: "encryptionkey@0"
I20260929 10:54:19.056035 10154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:54:19.070701 10154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:54:19.072721 10154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:54:19.072885 10154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:54:19.076706 10154 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:44521
I20260929 10:54:19.076745 10222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:44521 every 8 connection(s)
I20260929 10:54:19.077162 10154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:54:19.077973 10223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:54:19.080245 10223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f794235d908a4b179f7d0fcd8ee4f39f: Bootstrap starting.
I20260929 10:54:19.080859 10223 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f794235d908a4b179f7d0fcd8ee4f39f: Neither blocks nor log segments found. Creating new log.
I20260929 10:54:19.081176 10223 log.cc:824] T 00000000000000000000000000000000 P f794235d908a4b179f7d0fcd8ee4f39f: Log is configured to *not* fsync() on all Append() calls
I20260929 10:54:19.081853 10223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f794235d908a4b179f7d0fcd8ee4f39f: No bootstrap required, opened a new log
I20260929 10:54:19.083423 10223 raft_consensus.cc:348] T 00000000000000000000000000000000 P f794235d908a4b179f7d0fcd8ee4f39f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f794235d908a4b179f7d0fcd8ee4f39f" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44521 } }
I20260929 10:54:19.083554 10223 raft_consensus.cc:374] T 00000000000000000000000000000000 P f794235d908a4b179f7d0fcd8ee4f39f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:54:19.083598 10223 raft_consensus.cc:729] T 00000000000000000000000000000000 P f794235d908a4b179f7d0fcd8ee4f39f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f794235d908a4b179f7d0fcd8ee4f39f, State: Initialized, Role: FOLLOWER
I20260929 10:54:19.083760 10223 consensus_queue.cc:260] T 00000000000000000000000000000000 P f794235d908a4b179f7d0fcd8ee4f39f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f794235d908a4b179f7d0fcd8ee4f39f" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44521 } }
I20260929 10:54:19.083841 10223 raft_consensus.cc:388] T 00000000000000000000000000000000 P f794235d908a4b179f7d0fcd8ee4f39f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:54:19.083890 10223 raft_consensus.cc:482] T 00000000000000000000000000000000 P f794235d908a4b179f7d0fcd8ee4f39f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:54:19.083942 10223 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f794235d908a4b179f7d0fcd8ee4f39f [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:54:19.084677 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 10154
I20260929 10:54:19.084816 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:54:19.084831 10223 raft_consensus.cc:504] T 00000000000000000000000000000000 P f794235d908a4b179f7d0fcd8ee4f39f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f794235d908a4b179f7d0fcd8ee4f39f" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44521 } }
I20260929 10:54:19.084955 10223 leader_election.cc:302] T 00000000000000000000000000000000 P f794235d908a4b179f7d0fcd8ee4f39f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f794235d908a4b179f7d0fcd8ee4f39f; no voters:
I20260929 10:54:19.085028 20803 external_mini_cluster.cc:1428] Setting key d31b6dfca63bd9ef0b2459b6e3fb6896
I20260929 10:54:19.085132 10223 leader_election.cc:288] T 00000000000000000000000000000000 P f794235d908a4b179f7d0fcd8ee4f39f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:54:19.085156 10226 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f794235d908a4b179f7d0fcd8ee4f39f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:54:19.085263 10226 raft_consensus.cc:686] T 00000000000000000000000000000000 P f794235d908a4b179f7d0fcd8ee4f39f [term 1 LEADER]: Becoming Leader. State: Replica: f794235d908a4b179f7d0fcd8ee4f39f, State: Running, Role: LEADER
I20260929 10:54:19.085335 10226 consensus_queue.cc:237] T 00000000000000000000000000000000 P f794235d908a4b179f7d0fcd8ee4f39f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f794235d908a4b179f7d0fcd8ee4f39f" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44521 } }
I20260929 10:54:19.085398 10223 sys_catalog.cc:564] T 00000000000000000000000000000000 P f794235d908a4b179f7d0fcd8ee4f39f [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:54:19.085624 10226 sys_catalog.cc:455] T 00000000000000000000000000000000 P f794235d908a4b179f7d0fcd8ee4f39f [sys.catalog]: SysCatalogTable state changed. Reason: New leader f794235d908a4b179f7d0fcd8ee4f39f. Latest consensus state: current_term: 1 leader_uuid: "f794235d908a4b179f7d0fcd8ee4f39f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f794235d908a4b179f7d0fcd8ee4f39f" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44521 } } }
I20260929 10:54:19.085681 10226 sys_catalog.cc:458] T 00000000000000000000000000000000 P f794235d908a4b179f7d0fcd8ee4f39f [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:19.085826 10227 sys_catalog.cc:455] T 00000000000000000000000000000000 P f794235d908a4b179f7d0fcd8ee4f39f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f794235d908a4b179f7d0fcd8ee4f39f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f794235d908a4b179f7d0fcd8ee4f39f" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 44521 } } }
I20260929 10:54:19.085902 10227 sys_catalog.cc:458] T 00000000000000000000000000000000 P f794235d908a4b179f7d0fcd8ee4f39f [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:19.086083 10232 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:54:19.086621 10232 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:54:19.088475 10232 catalog_manager.cc:1347] Generated new cluster ID: 457c63e9b10f48efbb3e2dd2874a1864
I20260929 10:54:19.088533 10232 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:54:19.091399 10231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:19.085589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53871 (local address 127.20.80.254:44521)
0929 10:54:19.085899 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:19.085909 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:19.085943 (+ 34us) server_negotiation.cc:407] Connection header received
0929 10:54:19.086063 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:19.086070 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:19.086130 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:19.086270 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:19.086519 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:19.088001 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:19.089143 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:19.089745 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:19.089842 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:19.090622 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:19.090626 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:19.090626 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:54:19.090907 (+ 281us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:19.090955 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:54:19.090971 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:19.091065 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:19.091174 (+ 109us) server_negotiation.cc:299] Negotiation successful
0929 10:54:19.091200 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":66,"threads_started":1}
I20260929 10:54:19.092303 20803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:54:19.097654 10232 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:54:19.098336 10232 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:54:19.100659 10231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:19.093870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35431 (local address 127.20.80.254:44521)
0929 10:54:19.093906 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:19.093907 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:19.097194 (+ 3287us) server_negotiation.cc:407] Connection header received
0929 10:54:19.097294 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:19.097296 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:19.097309 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:19.097369 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:19.097538 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:19.098287 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:19.099340 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:19.099666 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:19.099739 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:19.100424 (+ 685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:19.100426 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:19.100426 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:54:19.100431 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:19.100446 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:54:19.100451 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:19.100517 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:19.100573 (+ 56us) server_negotiation.cc:299] Negotiation successful
0929 10:54:19.100585 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 10:54:19.102452 10232 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f794235d908a4b179f7d0fcd8ee4f39f: Generated new TSK 0
I20260929 10:54:19.102612 10232 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:54:19.108575 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 10154
W20260929 10:54:19.113327 10139 connection.cc:535] client connection to 127.20.80.254:44521 recv error: Network error: failed to read from TLS socket (remote: 127.20.80.254:44521): Connection reset by peer (error 104)
2026-09-29T10:54:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1326 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:54:19 dist-test-slave-vqch krb5kdc[10255](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:54:19 dist-test-slave-vqch krb5kdc[10255](info): set up 2 sockets
Sep 29 10:54:19 dist-test-slave-vqch krb5kdc[10255](info): commencing operation
krb5kdc: starting...
W20260929 10:54:20.148643 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:54:20 dist-test-slave-vqch krb5kdc[10255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:54:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:54:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.80.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
I20260929 10:54:20.268026 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:33253
--webserver_interface=127.20.80.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:35191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:33253
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:54:20.347079 10268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:54:20.347326 10268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:54:20.347365 10268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:54:20.348508 10268 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:54:20.348572 10268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:54:20.348598 10268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:54:20.348618 10268 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:54:20.348644 10268 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:54:20.350108 10268 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:35191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:33253
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:33253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:54:20.350375 10268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:54:20.350592 10268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:54:20.355324 10273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:20.355334 10274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:20.355460 10268 server_base.cc:1034] running on GCE node
W20260929 10:54:20.355334 10276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:20.355845 10268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:54:20.356122 10268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:54:20.357272 10268 hybrid_clock.cc:648] HybridClock initialized: now 1790679260357273 us; error 43 us; skew 500 ppm
Sep 29 10:54:20 dist-test-slave-vqch krb5kdc[10255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679260, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:54:20.359663 10268 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:54:20.360107 10268 webserver.cc:460] Webserver started at http://127.20.80.254:38673/ using document root <none> and password file <none>
I20260929 10:54:20.360317 10268 fs_manager.cc:359] Metadata directory not provided
I20260929 10:54:20.360380 10268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:54:20.360520 10268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:54:20.361583 10268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "c4a468eb559c4f8699ef983764036fbf"
format_stamp: "Formatted at 2026-09-29 10:54:20 on dist-test-slave-vqch"
server_key: "1defdda0b0f6200cac6f15f78273edff"
server_key_iv: "91834963a120f18a3b4c6bba2837c63e"
server_key_version: "encryptionkey@0"
I20260929 10:54:20.361943 10268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "c4a468eb559c4f8699ef983764036fbf"
format_stamp: "Formatted at 2026-09-29 10:54:20 on dist-test-slave-vqch"
server_key: "1defdda0b0f6200cac6f15f78273edff"
server_key_iv: "91834963a120f18a3b4c6bba2837c63e"
server_key_version: "encryptionkey@0"
I20260929 10:54:20.363243 10268 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:54:20.363976 10283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:20.364166 10268 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:54:20.364260 10268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "c4a468eb559c4f8699ef983764036fbf"
format_stamp: "Formatted at 2026-09-29 10:54:20 on dist-test-slave-vqch"
server_key: "1defdda0b0f6200cac6f15f78273edff"
server_key_iv: "91834963a120f18a3b4c6bba2837c63e"
server_key_version: "encryptionkey@0"
I20260929 10:54:20.364333 10268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:54:20.367511 10268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:54:20.369565 10268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:54:20.369724 10268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:54:20.376581 10336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:33253 every 8 connection(s)
I20260929 10:54:20.376758 10268 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:33253
I20260929 10:54:20.377225 10268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:54:20.378436 10337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:54:20.381786 10337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4a468eb559c4f8699ef983764036fbf: Bootstrap starting.
I20260929 10:54:20.382390 10337 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4a468eb559c4f8699ef983764036fbf: Neither blocks nor log segments found. Creating new log.
I20260929 10:54:20.382496 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 10268
I20260929 10:54:20.382603 10337 log.cc:824] T 00000000000000000000000000000000 P c4a468eb559c4f8699ef983764036fbf: Log is configured to *not* fsync() on all Append() calls
I20260929 10:54:20.382607 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:54:20.382810 20803 external_mini_cluster.cc:1428] Setting key 37c5f78a9adc0a2686453fdda859c7d5
I20260929 10:54:20.383366 10337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4a468eb559c4f8699ef983764036fbf: No bootstrap required, opened a new log
I20260929 10:54:20.385807 10341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:20.383322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54233 (local address 127.20.80.254:33253)
0929 10:54:20.383585 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:20.383592 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:20.383615 (+ 23us) server_negotiation.cc:407] Connection header received
0929 10:54:20.384110 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:20.384119 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:20.384157 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:20.384606 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:20.385138 (+ 532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:20.385142 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:20.385142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:54:20.385147 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:20.385191 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:54:20.385216 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:20.385523 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:20.385556 (+ 33us) server_negotiation.cc:299] Negotiation successful
0929 10:54:20.385599 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":69,"threads_started":1}
I20260929 10:54:20.387019 20803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:54:20.386999 10337 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4a468eb559c4f8699ef983764036fbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4a468eb559c4f8699ef983764036fbf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 33253 } }
I20260929 10:54:20.387166 10337 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4a468eb559c4f8699ef983764036fbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:54:20.387205 10337 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4a468eb559c4f8699ef983764036fbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4a468eb559c4f8699ef983764036fbf, State: Initialized, Role: FOLLOWER
I20260929 10:54:20.387315 10337 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4a468eb559c4f8699ef983764036fbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4a468eb559c4f8699ef983764036fbf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 33253 } }
I20260929 10:54:20.387387 10337 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4a468eb559c4f8699ef983764036fbf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:54:20.387413 10337 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4a468eb559c4f8699ef983764036fbf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:54:20.387460 10337 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4a468eb559c4f8699ef983764036fbf [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:54:20.388201 10337 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4a468eb559c4f8699ef983764036fbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4a468eb559c4f8699ef983764036fbf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 33253 } }
I20260929 10:54:20.388326 10337 leader_election.cc:302] T 00000000000000000000000000000000 P c4a468eb559c4f8699ef983764036fbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4a468eb559c4f8699ef983764036fbf; no voters:
I20260929 10:54:20.388485 10337 leader_election.cc:288] T 00000000000000000000000000000000 P c4a468eb559c4f8699ef983764036fbf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:54:20.388554 10346 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4a468eb559c4f8699ef983764036fbf [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:54:20.388728 10337 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4a468eb559c4f8699ef983764036fbf [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:54:20.388743 10346 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4a468eb559c4f8699ef983764036fbf [term 1 LEADER]: Becoming Leader. State: Replica: c4a468eb559c4f8699ef983764036fbf, State: Running, Role: LEADER
I20260929 10:54:20.388846 10346 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4a468eb559c4f8699ef983764036fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4a468eb559c4f8699ef983764036fbf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 33253 } }
I20260929 10:54:20.389647 10347 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4a468eb559c4f8699ef983764036fbf [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4a468eb559c4f8699ef983764036fbf. Latest consensus state: current_term: 1 leader_uuid: "c4a468eb559c4f8699ef983764036fbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4a468eb559c4f8699ef983764036fbf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 33253 } } }
I20260929 10:54:20.389878 10347 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4a468eb559c4f8699ef983764036fbf [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:20.389804 10349 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4a468eb559c4f8699ef983764036fbf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4a468eb559c4f8699ef983764036fbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4a468eb559c4f8699ef983764036fbf" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 33253 } } }
I20260929 10:54:20.390158 10349 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4a468eb559c4f8699ef983764036fbf [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:20.390251 10341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:20.388633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34237 (local address 127.20.80.254:33253)
0929 10:54:20.388667 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:20.388668 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:20.389145 (+ 477us) server_negotiation.cc:407] Connection header received
0929 10:54:20.389251 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:20.389252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:20.389257 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:20.389287 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:20.389976 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:20.389977 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:20.389978 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:54:20.389983 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:20.389998 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:54:20.390004 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:20.390036 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:20.390167 (+ 131us) server_negotiation.cc:299] Negotiation successful
0929 10:54:20.390181 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 10:54:20.390556 10354 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:54:20.391105 10354 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:54:20.392730 10354 catalog_manager.cc:1347] Generated new cluster ID: 9c5a5c9be4854332ba3d870b67aef1fb
I20260929 10:54:20.392784 10354 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:54:20.425756 10354 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:54:20.425945 10354 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:54:20.430557 10354 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4a468eb559c4f8699ef983764036fbf: Generated new TSK 0
I20260929 10:54:20.430768 10354 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:54:20.441610 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 10268
2026-09-29T10:54:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1333 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:54:20 dist-test-slave-vqch krb5kdc[10368](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:54:20 dist-test-slave-vqch krb5kdc[10368](info): set up 2 sockets
Sep 29 10:54:20 dist-test-slave-vqch krb5kdc[10368](info): commencing operation
krb5kdc: starting...
W20260929 10:54:21.509547 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:54:21 dist-test-slave-vqch krb5kdc[10368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:54:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:54:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.80.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
I20260929 10:54:21.628579 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:42451
--webserver_interface=127.20.80.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:46635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:42451
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:54:21.709362 10381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:54:21.709596 10381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:54:21.709635 10381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:54:21.710762 10381 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:54:21.710824 10381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:54:21.710850 10381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:54:21.710875 10381 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:54:21.710904 10381 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:54:21.712376 10381 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:46635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:42451
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:42451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:54:21.712641 10381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:54:21.712862 10381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:54:21.717634 10386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:21.717677 10381 server_base.cc:1034] running on GCE node
W20260929 10:54:21.717694 10387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:21.717634 10389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:21.718103 10381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:54:21.718430 10381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:54:21.719645 10381 hybrid_clock.cc:648] HybridClock initialized: now 1790679261719638 us; error 39 us; skew 500 ppm
Sep 29 10:54:21 dist-test-slave-vqch krb5kdc[10368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679261, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:54:21.722378 10381 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:54:21.722870 10381 webserver.cc:460] Webserver started at http://127.20.80.254:33861/ using document root <none> and password file <none>
I20260929 10:54:21.723075 10381 fs_manager.cc:359] Metadata directory not provided
I20260929 10:54:21.723124 10381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:54:21.723219 10381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:54:21.724283 10381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "38232a31852e4bcb80dc527da87e7506"
format_stamp: "Formatted at 2026-09-29 10:54:21 on dist-test-slave-vqch"
server_key: "90c1ec8c936f2cd228a88ec36ebe524f"
server_key_iv: "3c85f3aac5152af5547a9bdc6250dd04"
server_key_version: "encryptionkey@0"
I20260929 10:54:21.724613 10381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "38232a31852e4bcb80dc527da87e7506"
format_stamp: "Formatted at 2026-09-29 10:54:21 on dist-test-slave-vqch"
server_key: "90c1ec8c936f2cd228a88ec36ebe524f"
server_key_iv: "3c85f3aac5152af5547a9bdc6250dd04"
server_key_version: "encryptionkey@0"
I20260929 10:54:21.726094 10381 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:54:21.726846 10397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:21.727046 10381 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:54:21.727124 10381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "38232a31852e4bcb80dc527da87e7506"
format_stamp: "Formatted at 2026-09-29 10:54:21 on dist-test-slave-vqch"
server_key: "90c1ec8c936f2cd228a88ec36ebe524f"
server_key_iv: "3c85f3aac5152af5547a9bdc6250dd04"
server_key_version: "encryptionkey@0"
I20260929 10:54:21.727180 10381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:54:21.745132 10381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:54:21.747256 10381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:54:21.747380 10381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:54:21.751919 10381 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:42451
I20260929 10:54:21.751986 10449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:42451 every 8 connection(s)
I20260929 10:54:21.752353 10381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:54:21.753316 10450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:54:21.753471 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 10381
I20260929 10:54:21.753583 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:54:21.753773 20803 external_mini_cluster.cc:1428] Setting key baebc6a6b94506f80282a4e944947865
I20260929 10:54:21.756798 10450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38232a31852e4bcb80dc527da87e7506: Bootstrap starting.
I20260929 10:54:21.757575 10450 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38232a31852e4bcb80dc527da87e7506: Neither blocks nor log segments found. Creating new log.
I20260929 10:54:21.757877 10450 log.cc:824] T 00000000000000000000000000000000 P 38232a31852e4bcb80dc527da87e7506: Log is configured to *not* fsync() on all Append() calls
I20260929 10:54:21.758711 10450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38232a31852e4bcb80dc527da87e7506: No bootstrap required, opened a new log
I20260929 10:54:21.760840 10450 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38232a31852e4bcb80dc527da87e7506 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38232a31852e4bcb80dc527da87e7506" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 42451 } }
I20260929 10:54:21.761047 10450 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38232a31852e4bcb80dc527da87e7506 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:54:21.761085 10450 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38232a31852e4bcb80dc527da87e7506 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38232a31852e4bcb80dc527da87e7506, State: Initialized, Role: FOLLOWER
Sep 29 10:54:21 dist-test-slave-vqch krb5kdc[10368](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:54:21.761228 10450 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38232a31852e4bcb80dc527da87e7506 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38232a31852e4bcb80dc527da87e7506" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 42451 } }
I20260929 10:54:21.761328 10450 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38232a31852e4bcb80dc527da87e7506 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:54:21.761374 10450 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38232a31852e4bcb80dc527da87e7506 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:54:21.761444 10450 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38232a31852e4bcb80dc527da87e7506 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:54:21.762020 10450 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38232a31852e4bcb80dc527da87e7506 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38232a31852e4bcb80dc527da87e7506" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 42451 } }
I20260929 10:54:21.762141 10450 leader_election.cc:302] T 00000000000000000000000000000000 P 38232a31852e4bcb80dc527da87e7506 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38232a31852e4bcb80dc527da87e7506; no voters:
I20260929 10:54:21.762316 10450 leader_election.cc:288] T 00000000000000000000000000000000 P 38232a31852e4bcb80dc527da87e7506 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:54:21.762399 10455 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38232a31852e4bcb80dc527da87e7506 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:54:21.762591 10450 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38232a31852e4bcb80dc527da87e7506 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:54:21.762634 10455 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38232a31852e4bcb80dc527da87e7506 [term 1 LEADER]: Becoming Leader. State: Replica: 38232a31852e4bcb80dc527da87e7506, State: Running, Role: LEADER
I20260929 10:54:21.763141 10455 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38232a31852e4bcb80dc527da87e7506 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38232a31852e4bcb80dc527da87e7506" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 42451 } }
I20260929 10:54:21.763568 10457 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38232a31852e4bcb80dc527da87e7506 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38232a31852e4bcb80dc527da87e7506. Latest consensus state: current_term: 1 leader_uuid: "38232a31852e4bcb80dc527da87e7506" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38232a31852e4bcb80dc527da87e7506" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 42451 } } }
I20260929 10:54:21.763739 10457 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38232a31852e4bcb80dc527da87e7506 [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:21.763762 10456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38232a31852e4bcb80dc527da87e7506 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38232a31852e4bcb80dc527da87e7506" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38232a31852e4bcb80dc527da87e7506" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 42451 } } }
I20260929 10:54:21.763875 10456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38232a31852e4bcb80dc527da87e7506 [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:21.764045 10468 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:54:21.764771 10468 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:54:21.766304 10468 catalog_manager.cc:1347] Generated new cluster ID: 01f852472d814c659c4ad7ffbd12ac01
I20260929 10:54:21.766316 10453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:21.754377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52745 (local address 127.20.80.254:42451)
0929 10:54:21.754533 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:21.754547 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:21.754575 (+ 28us) server_negotiation.cc:407] Connection header received
0929 10:54:21.755041 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:21.755047 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:21.755097 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:21.755229 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:21.756053 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:21.757097 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:21.758220 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:21.758792 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:21.758946 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:21.761757 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:21.761762 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:21.761763 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:21.761775 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:21.764178 (+ 2403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:21.764664 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:21.764667 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:21.764667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:21.764771 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:21.765071 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:21.765072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:21.765073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:21.765394 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:21.765542 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:21.765981 (+ 439us) server_negotiation.cc:299] Negotiation successful
0929 10:54:21.766126 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":76,"threads_started":1}
I20260929 10:54:21.766362 10468 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:54:21.767204 20803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:54:21.776291 10453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:21.768775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56765 (local address 127.20.80.254:42451)
0929 10:54:21.768830 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:21.768832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:21.768948 (+ 116us) server_negotiation.cc:407] Connection header received
0929 10:54:21.769019 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:21.769020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:21.769031 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:21.769100 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:21.769797 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:21.770577 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:21.771566 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:21.771918 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:21.771993 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:21.773486 (+ 1493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:21.773487 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:21.773487 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:21.773494 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:21.775135 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:21.775512 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:21.775513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:21.775513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:21.775576 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:21.775776 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:21.775777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:21.775777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:21.775907 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:21.776006 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:21.776124 (+ 118us) server_negotiation.cc:299] Negotiation successful
0929 10:54:21.776199 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 10:54:21.778164 10468 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:54:21.778729 10468 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:54:21.784526 10468 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38232a31852e4bcb80dc527da87e7506: Generated new TSK 0
I20260929 10:54:21.784719 10468 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:54:21.786761 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 10381
2026-09-29T10:54:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1345 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:54:21 dist-test-slave-vqch krb5kdc[10481](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:54:21 dist-test-slave-vqch krb5kdc[10481](info): set up 2 sockets
Sep 29 10:54:21 dist-test-slave-vqch krb5kdc[10481](info): commencing operation
krb5kdc: starting...
W20260929 10:54:22.892959 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:54:22 dist-test-slave-vqch krb5kdc[10481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:54:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:54:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.80.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
I20260929 10:54:23.015566 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:33525
--webserver_interface=127.20.80.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:37763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:33525
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:54:23.096067 10494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:54:23.096287 10494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:54:23.096310 10494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:54:23.097524 10494 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:54:23.097571 10494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:54:23.097585 10494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:54:23.097599 10494 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:54:23.097610 10494 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:54:23.099110 10494 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:37763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:33525
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:33525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:54:23.099362 10494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:54:23.099604 10494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:54:23.104637 10502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:23.104624 10499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:23.104707 10500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:23.104910 10494 server_base.cc:1034] running on GCE node
I20260929 10:54:23.105161 10494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:54:23.105459 10494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:54:23.106631 10494 hybrid_clock.cc:648] HybridClock initialized: now 1790679263106627 us; error 37 us; skew 500 ppm
Sep 29 10:54:23 dist-test-slave-vqch krb5kdc[10481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679263, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:54:23.114495 10494 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:54:23.114946 10494 webserver.cc:460] Webserver started at http://127.20.80.254:37361/ using document root <none> and password file <none>
I20260929 10:54:23.115175 10494 fs_manager.cc:359] Metadata directory not provided
I20260929 10:54:23.115252 10494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:54:23.115402 10494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:54:23.116482 10494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "6f2520c25510413683f5d638ee7ecc0e"
format_stamp: "Formatted at 2026-09-29 10:54:23 on dist-test-slave-vqch"
server_key: "c9f27c01968f2871b8979c111a310fda"
server_key_iv: "62ec703d0251abea2811d7cf86599f1b"
server_key_version: "encryptionkey@0"
I20260929 10:54:23.116850 10494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "6f2520c25510413683f5d638ee7ecc0e"
format_stamp: "Formatted at 2026-09-29 10:54:23 on dist-test-slave-vqch"
server_key: "c9f27c01968f2871b8979c111a310fda"
server_key_iv: "62ec703d0251abea2811d7cf86599f1b"
server_key_version: "encryptionkey@0"
I20260929 10:54:23.118223 10494 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:54:23.118947 10509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:23.119118 10494 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:54:23.119251 10494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "6f2520c25510413683f5d638ee7ecc0e"
format_stamp: "Formatted at 2026-09-29 10:54:23 on dist-test-slave-vqch"
server_key: "c9f27c01968f2871b8979c111a310fda"
server_key_iv: "62ec703d0251abea2811d7cf86599f1b"
server_key_version: "encryptionkey@0"
I20260929 10:54:23.119328 10494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:54:23.139922 10494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:54:23.142318 10494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:54:23.142504 10494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:54:23.146822 10494 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:33525
I20260929 10:54:23.146860 10562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:33525 every 8 connection(s)
I20260929 10:54:23.147223 10494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:54:23.148097 10563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:54:23.150487 10563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f2520c25510413683f5d638ee7ecc0e: Bootstrap starting.
I20260929 10:54:23.150700 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 10494
I20260929 10:54:23.150822 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:54:23.151011 20803 external_mini_cluster.cc:1428] Setting key e3d8562bbca5025b92bdb63b301b25f0
I20260929 10:54:23.151238 10563 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f2520c25510413683f5d638ee7ecc0e: Neither blocks nor log segments found. Creating new log.
I20260929 10:54:23.151597 10563 log.cc:824] T 00000000000000000000000000000000 P 6f2520c25510413683f5d638ee7ecc0e: Log is configured to *not* fsync() on all Append() calls
I20260929 10:54:23.152415 10563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f2520c25510413683f5d638ee7ecc0e: No bootstrap required, opened a new log
I20260929 10:54:23.154846 10563 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f2520c25510413683f5d638ee7ecc0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2520c25510413683f5d638ee7ecc0e" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 33525 } }
I20260929 10:54:23.155018 10563 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f2520c25510413683f5d638ee7ecc0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:54:23.155066 10563 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f2520c25510413683f5d638ee7ecc0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f2520c25510413683f5d638ee7ecc0e, State: Initialized, Role: FOLLOWER
I20260929 10:54:23.155200 10563 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f2520c25510413683f5d638ee7ecc0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2520c25510413683f5d638ee7ecc0e" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 33525 } }
I20260929 10:54:23.155300 10563 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f2520c25510413683f5d638ee7ecc0e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:54:23.155335 10563 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f2520c25510413683f5d638ee7ecc0e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:54:23.155381 10563 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f2520c25510413683f5d638ee7ecc0e [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:54:23.156109 10563 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f2520c25510413683f5d638ee7ecc0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2520c25510413683f5d638ee7ecc0e" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 33525 } }
I20260929 10:54:23.156225 10563 leader_election.cc:302] T 00000000000000000000000000000000 P 6f2520c25510413683f5d638ee7ecc0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f2520c25510413683f5d638ee7ecc0e; no voters:
I20260929 10:54:23.156399 10563 leader_election.cc:288] T 00000000000000000000000000000000 P 6f2520c25510413683f5d638ee7ecc0e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:54:23.156490 10568 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f2520c25510413683f5d638ee7ecc0e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:54:23.156667 10563 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f2520c25510413683f5d638ee7ecc0e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:54:23.156941 10568 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f2520c25510413683f5d638ee7ecc0e [term 1 LEADER]: Becoming Leader. State: Replica: 6f2520c25510413683f5d638ee7ecc0e, State: Running, Role: LEADER
I20260929 10:54:23.157110 10568 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f2520c25510413683f5d638ee7ecc0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2520c25510413683f5d638ee7ecc0e" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 33525 } }
I20260929 10:54:23.157557 10570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f2520c25510413683f5d638ee7ecc0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f2520c25510413683f5d638ee7ecc0e. Latest consensus state: current_term: 1 leader_uuid: "6f2520c25510413683f5d638ee7ecc0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2520c25510413683f5d638ee7ecc0e" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 33525 } } }
I20260929 10:54:23.157649 10570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f2520c25510413683f5d638ee7ecc0e [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:23.157872 10569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f2520c25510413683f5d638ee7ecc0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f2520c25510413683f5d638ee7ecc0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2520c25510413683f5d638ee7ecc0e" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 33525 } } }
I20260929 10:54:23.157943 10569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f2520c25510413683f5d638ee7ecc0e [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:23.157951 10566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:23.151581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42103 (local address 127.20.80.254:33525)
0929 10:54:23.151807 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:23.151817 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:23.151852 (+ 35us) server_negotiation.cc:407] Connection header received
0929 10:54:23.152429 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:23.152438 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:23.152503 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:23.152657 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:23.152875 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:23.154345 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:23.155269 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:23.155900 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:23.156019 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:23.156896 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:23.156900 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:23.156901 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:54:23.157252 (+ 351us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:23.157299 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:54:23.157317 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:23.157592 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:23.157626 (+ 34us) server_negotiation.cc:299] Negotiation successful
0929 10:54:23.157703 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":82,"threads_started":1}
I20260929 10:54:23.158406 10578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:54:23.158921 10578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:54:23.159312 20803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:54:23.162055 10578 catalog_manager.cc:1347] Generated new cluster ID: 531f9d5459d14250842cd85a82afc219
I20260929 10:54:23.162134 10578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:54:23.175132 10566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:23.160716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37057 (local address 127.20.80.254:33525)
0929 10:54:23.160763 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:23.160765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:23.160896 (+ 131us) server_negotiation.cc:407] Connection header received
0929 10:54:23.160968 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:23.160968 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:23.161016 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:23.161117 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:23.161289 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:23.161780 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:23.173644 (+ 11864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:23.173954 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:23.174022 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:23.174849 (+ 827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:23.174851 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:23.174851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:54:23.174857 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:23.174874 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:54:23.174879 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:23.174930 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:23.175008 (+ 78us) server_negotiation.cc:299] Negotiation successful
0929 10:54:23.175036 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 10:54:23.179244 10578 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:54:23.180063 10578 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:54:23.185166 10578 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f2520c25510413683f5d638ee7ecc0e: Generated new TSK 0
I20260929 10:54:23.185375 10578 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:54:23.194682 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 10494
2026-09-29T10:54:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1407 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:54:23 dist-test-slave-vqch krb5kdc[10594](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:54:23 dist-test-slave-vqch krb5kdc[10594](info): set up 2 sockets
Sep 29 10:54:23 dist-test-slave-vqch krb5kdc[10594](info): commencing operation
krb5kdc: starting...
W20260929 10:54:24.249686 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:54:24 dist-test-slave-vqch krb5kdc[10594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:54:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:54:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.80.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
I20260929 10:54:24.372925 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:43341
--webserver_interface=127.20.80.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:37447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:43341
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:54:24.453115 10607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:54:24.453354 10607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:54:24.453392 10607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:54:24.454535 10607 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:54:24.454607 10607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:54:24.454633 10607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:54:24.454653 10607 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:54:24.454689 10607 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:54:24.456131 10607 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:37447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:43341
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:43341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:54:24.456405 10607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:54:24.456636 10607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:54:24.461359 10613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:24.461359 10612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:24.461444 10615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:24.461547 10607 server_base.cc:1034] running on GCE node
I20260929 10:54:24.461887 10607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:54:24.462138 10607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:54:24.463321 10607 hybrid_clock.cc:648] HybridClock initialized: now 1790679264463298 us; error 43 us; skew 500 ppm
Sep 29 10:54:24 dist-test-slave-vqch krb5kdc[10594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679264, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:54:24.465955 10607 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:54:24.466390 10607 webserver.cc:460] Webserver started at http://127.20.80.254:38299/ using document root <none> and password file <none>
I20260929 10:54:24.466624 10607 fs_manager.cc:359] Metadata directory not provided
I20260929 10:54:24.466701 10607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:54:24.466854 10607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:54:24.468029 10607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "8b9248007b4a4198823c8b49cc9c7471"
format_stamp: "Formatted at 2026-09-29 10:54:24 on dist-test-slave-vqch"
server_key: "214f002cb93b70a74a9a55c05ed45dbe"
server_key_iv: "b1270961e3c107708ff49b11782ea83b"
server_key_version: "encryptionkey@0"
I20260929 10:54:24.468396 10607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "8b9248007b4a4198823c8b49cc9c7471"
format_stamp: "Formatted at 2026-09-29 10:54:24 on dist-test-slave-vqch"
server_key: "214f002cb93b70a74a9a55c05ed45dbe"
server_key_iv: "b1270961e3c107708ff49b11782ea83b"
server_key_version: "encryptionkey@0"
I20260929 10:54:24.469842 10607 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260929 10:54:24.470566 10622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:24.470772 10607 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:54:24.470897 10607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "8b9248007b4a4198823c8b49cc9c7471"
format_stamp: "Formatted at 2026-09-29 10:54:24 on dist-test-slave-vqch"
server_key: "214f002cb93b70a74a9a55c05ed45dbe"
server_key_iv: "b1270961e3c107708ff49b11782ea83b"
server_key_version: "encryptionkey@0"
I20260929 10:54:24.470988 10607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:54:24.474761 10607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:54:24.477438 10607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:54:24.477602 10607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:54:24.481968 10607 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:43341
I20260929 10:54:24.482009 10675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:43341 every 8 connection(s)
I20260929 10:54:24.482355 10607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:54:24.483307 10676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:54:24.485584 10676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b9248007b4a4198823c8b49cc9c7471: Bootstrap starting.
I20260929 10:54:24.486146 10676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b9248007b4a4198823c8b49cc9c7471: Neither blocks nor log segments found. Creating new log.
I20260929 10:54:24.486426 10676 log.cc:824] T 00000000000000000000000000000000 P 8b9248007b4a4198823c8b49cc9c7471: Log is configured to *not* fsync() on all Append() calls
I20260929 10:54:24.487097 10676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b9248007b4a4198823c8b49cc9c7471: No bootstrap required, opened a new log
I20260929 10:54:24.487555 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 10607
I20260929 10:54:24.487634 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:54:24.487916 20803 external_mini_cluster.cc:1428] Setting key 0b652a0693115a8d60b07fea74fe7794
I20260929 10:54:24.489794 10676 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b9248007b4a4198823c8b49cc9c7471 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b9248007b4a4198823c8b49cc9c7471" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 43341 } }
I20260929 10:54:24.489967 10676 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b9248007b4a4198823c8b49cc9c7471 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:54:24.490011 10676 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b9248007b4a4198823c8b49cc9c7471 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b9248007b4a4198823c8b49cc9c7471, State: Initialized, Role: FOLLOWER
I20260929 10:54:24.490196 10676 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b9248007b4a4198823c8b49cc9c7471 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b9248007b4a4198823c8b49cc9c7471" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 43341 } }
I20260929 10:54:24.490310 10676 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b9248007b4a4198823c8b49cc9c7471 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:54:24.490357 10676 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b9248007b4a4198823c8b49cc9c7471 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:54:24.490408 10676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b9248007b4a4198823c8b49cc9c7471 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:54:24.491034 10680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:24.488412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57751 (local address 127.20.80.254:43341)
0929 10:54:24.488616 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:24.488626 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:24.488660 (+ 34us) server_negotiation.cc:407] Connection header received
0929 10:54:24.489739 (+ 1079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:24.489748 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:24.489783 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:24.489930 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:24.490590 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:24.490593 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:24.490593 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:54:24.490597 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:24.490631 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:54:24.490641 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:24.490696 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:24.490807 (+ 111us) server_negotiation.cc:299] Negotiation successful
0929 10:54:24.490867 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":102,"threads_started":1}
I20260929 10:54:24.491074 10676 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b9248007b4a4198823c8b49cc9c7471 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b9248007b4a4198823c8b49cc9c7471" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 43341 } }
I20260929 10:54:24.491184 10676 leader_election.cc:302] T 00000000000000000000000000000000 P 8b9248007b4a4198823c8b49cc9c7471 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b9248007b4a4198823c8b49cc9c7471; no voters:
I20260929 10:54:24.491343 10676 leader_election.cc:288] T 00000000000000000000000000000000 P 8b9248007b4a4198823c8b49cc9c7471 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:54:24.491402 10681 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b9248007b4a4198823c8b49cc9c7471 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:54:24.491549 10681 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b9248007b4a4198823c8b49cc9c7471 [term 1 LEADER]: Becoming Leader. State: Replica: 8b9248007b4a4198823c8b49cc9c7471, State: Running, Role: LEADER
I20260929 10:54:24.491583 10676 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b9248007b4a4198823c8b49cc9c7471 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:54:24.491658 10681 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b9248007b4a4198823c8b49cc9c7471 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b9248007b4a4198823c8b49cc9c7471" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 43341 } }
I20260929 10:54:24.492043 20803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:54:24.492091 10682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b9248007b4a4198823c8b49cc9c7471 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b9248007b4a4198823c8b49cc9c7471" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b9248007b4a4198823c8b49cc9c7471" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 43341 } } }
I20260929 10:54:24.492162 10682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b9248007b4a4198823c8b49cc9c7471 [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:24.492326 10683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b9248007b4a4198823c8b49cc9c7471 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b9248007b4a4198823c8b49cc9c7471. Latest consensus state: current_term: 1 leader_uuid: "8b9248007b4a4198823c8b49cc9c7471" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b9248007b4a4198823c8b49cc9c7471" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 43341 } } }
I20260929 10:54:24.492369 10683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b9248007b4a4198823c8b49cc9c7471 [sys.catalog]: This master's current role is: LEADER
W20260929 10:54:24.493712 10696 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8b9248007b4a4198823c8b49cc9c7471: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:54:24.493816 10696 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:54:24.493922 10687 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:54:24.494124 10687 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:54:24.495056 10680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:24.493723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53047 (local address 127.20.80.254:43341)
0929 10:54:24.493755 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:24.493757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:24.494073 (+ 316us) server_negotiation.cc:407] Connection header received
0929 10:54:24.494138 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:24.494139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:24.494144 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:24.494212 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:24.494838 (+ 626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:24.494839 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:24.494839 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:54:24.494844 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:24.494858 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:54:24.494862 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:24.494890 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:24.494990 (+ 100us) server_negotiation.cc:299] Negotiation successful
0929 10:54:24.495002 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260929 10:54:24.495697 10687 catalog_manager.cc:1347] Generated new cluster ID: 83771cfad02e489d926371f9647cbc61
I20260929 10:54:24.495743 10687 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:54:24.506008 10687 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:54:24.506191 10687 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:54:24.517972 10687 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b9248007b4a4198823c8b49cc9c7471: Generated new TSK 0
I20260929 10:54:24.518179 10687 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:54:24.528367 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 10607
2026-09-29T10:54:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1333 ms)
[----------] 18 tests from AuthTokenIssuingTest (23905 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:54:24 dist-test-slave-vqch krb5kdc[10707](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:54:24 dist-test-slave-vqch krb5kdc[10707](info): set up 2 sockets
Sep 29 10:54:24 dist-test-slave-vqch krb5kdc[10707](info): commencing operation
krb5kdc: starting...
W20260929 10:54:25.561798 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:54:25 dist-test-slave-vqch krb5kdc[10707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:54:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:54:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.80.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
I20260929 10:54:25.679682 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:41381
--webserver_interface=127.20.80.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:43635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:41381,127.20.80.253:44085,127.20.80.252:36567
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:54:25.760049 10720 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 10:54:25.760264 10720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:54:25.760281 10720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:54:25.760301 10720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:54:25.761394 10720 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:54:25.761428 10720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:54:25.761441 10720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:54:25.761453 10720 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:54:25.761466 10720 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:54:25.762874 10720 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:43635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:41381,127.20.80.253:44085,127.20.80.252:36567
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:41381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:54:25.763062 10720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:54:25.763235 10720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:54:25.768026 10728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:25.767997 10726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:25.768010 10725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:25.768327 10720 server_base.cc:1034] running on GCE node
I20260929 10:54:25.768571 10720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:54:25.768887 10720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:54:25.770040 10720 hybrid_clock.cc:648] HybridClock initialized: now 1790679265770017 us; error 43 us; skew 500 ppm
Sep 29 10:54:25 dist-test-slave-vqch krb5kdc[10707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679265, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:54:25.772696 10720 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:54:25.773231 10720 webserver.cc:460] Webserver started at http://127.20.80.254:46201/ using document root <none> and password file <none>
I20260929 10:54:25.773451 10720 fs_manager.cc:359] Metadata directory not provided
I20260929 10:54:25.773516 10720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:54:25.773676 10720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:54:25.774734 10720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "0339829159b64354a6bf01b5324e1f79"
format_stamp: "Formatted at 2026-09-29 10:54:25 on dist-test-slave-vqch"
server_key: "4aec1f539f6f3e51f61dc88c603d0485"
server_key_iv: "87a1229a198602e50df1c2a117b9142b"
server_key_version: "encryptionkey@0"
I20260929 10:54:25.775097 10720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "0339829159b64354a6bf01b5324e1f79"
format_stamp: "Formatted at 2026-09-29 10:54:25 on dist-test-slave-vqch"
server_key: "4aec1f539f6f3e51f61dc88c603d0485"
server_key_iv: "87a1229a198602e50df1c2a117b9142b"
server_key_version: "encryptionkey@0"
I20260929 10:54:25.776407 10720 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 10:54:25.777195 10735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:25.777372 10720 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:54:25.777480 10720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "0339829159b64354a6bf01b5324e1f79"
format_stamp: "Formatted at 2026-09-29 10:54:25 on dist-test-slave-vqch"
server_key: "4aec1f539f6f3e51f61dc88c603d0485"
server_key_iv: "87a1229a198602e50df1c2a117b9142b"
server_key_version: "encryptionkey@0"
I20260929 10:54:25.777561 10720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:54:25.793200 10720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:54:25.795403 10720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:54:25.795559 10720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:54:25.799870 10720 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:41381
I20260929 10:54:25.799929 10787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:41381 every 8 connection(s)
I20260929 10:54:25.800284 10720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:54:25.801362 10788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:54:25.802930 10788 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.80.254" port: 41381 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:54:25.804352 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 10720
I20260929 10:54:25.804448 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:54:25.804682 20803 external_mini_cluster.cc:1428] Setting key 60c63579b545147bdc37e2a64a172eaf
Sep 29 10:54:25 dist-test-slave-vqch krb5kdc[10707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679265, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.80.253@KRBTEST.COM; defaulting to no policy
I20260929 10:54:25.813254 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:25.803649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:41381 (local address 127.0.0.1:42136)
0929 10:54:25.803844 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:25.803896 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 10:54:25.804433 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:54:25.804888 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:54:25.804891 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:54:25.805209 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:54:25.805366 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:25.805371 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:25.806404 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:25.806406 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:25.807063 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:25.807065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:25.807609 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:25.807610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:25.807675 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:25.808232 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:54:25.808240 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:54:25.809908 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:54:25.811711 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:25.811714 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:25.811716 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:25.811884 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:25.812113 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:25.812114 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:25.812114 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:25.812207 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:25.812587 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:54:25.812595 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:54:25.812663 (+ 68us) client_negotiation.cc:769] Sending connection context
0929 10:54:25.812800 (+ 137us) client_negotiation.cc:240] Negotiation successful
0929 10:54:25.813061 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":119,"thread_start_us":116,"threads_started":1}
I20260929 10:54:25.813256 10791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:25.803715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42136 (local address 127.20.80.254:41381)
0929 10:54:25.803881 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:25.803889 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:25.803995 (+ 106us) server_negotiation.cc:407] Connection header received
0929 10:54:25.804722 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:25.804725 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:25.804753 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:25.804881 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:25.805475 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:25.806290 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:25.807156 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:25.807475 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:25.807605 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:25.810015 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:25.810022 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:25.810024 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:25.810036 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:25.811621 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:25.811960 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:25.811964 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:25.811964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:25.812045 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:25.812265 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:25.812266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:25.812266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:25.812478 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:25.812543 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:25.812836 (+ 293us) server_negotiation.cc:299] Negotiation successful
0929 10:54:25.812957 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":93,"threads_started":1}
I20260929 10:54:25.814292 10788 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44085 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.253@KRBTEST.COM" created.
I20260929 10:54:25.814723 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:25.814532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:44085 (local address 127.0.0.1:49432)
0929 10:54:25.814633 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:25.814686 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.253:44085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260929 10:54:25.814852 10737 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.80.253:44085: connect: Connection refused (error 111)
I20260929 10:54:25.815280 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:25.815182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:44085 (local address 127.0.0.1:49442)
0929 10:54:25.815232 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:25.815258 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.253:44085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260929 10:54:25.815389 10788 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.80.253:44085: Network error: Client connection negotiation failed: client connection to 127.20.80.253:44085: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.253.keytab.
Entry for principal kudu/127.20.80.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.253.keytab.
I20260929 10:54:25.828555 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.253:44085
--webserver_interface=127.20.80.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.253.keytab
--principal=kudu/127.20.80.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:43635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:41381,127.20.80.253:44085,127.20.80.252:36567
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:54:25.864619 10788 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44085 } attempt: 1
I20260929 10:54:25.865185 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:25.865049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:44085 (local address 127.0.0.1:49448)
0929 10:54:25.865107 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:25.865150 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.253:44085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260929 10:54:25.865614 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:25.865526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:44085 (local address 127.0.0.1:49452)
0929 10:54:25.865571 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:25.865597 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.253:44085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260929 10:54:25.865721 10788 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.80.253:44085: Network error: Client connection negotiation failed: client connection to 127.20.80.253:44085: connect: Connection refused (error 111)
W20260929 10:54:25.908761 10795 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 10:54:25.908972 10795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:54:25.909004 10795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:54:25.909024 10795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:54:25.910130 10795 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:54:25.910177 10795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:54:25.910192 10795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:54:25.910204 10795 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:54:25.910216 10795 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:54:25.911686 10795 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:43635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:41381,127.20.80.253:44085,127.20.80.252:36567
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.253.keytab
--principal=kudu/127.20.80.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.253:44085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.80.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:54:25.911922 10795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:54:25.912148 10795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:54:25.917268 10801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:25.917315 10803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:25.917258 10800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:25.917523 10795 server_base.cc:1034] running on GCE node
I20260929 10:54:25.917865 10795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:54:25.918233 10795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:54:25.919415 10795 hybrid_clock.cc:648] HybridClock initialized: now 1790679265919392 us; error 39 us; skew 500 ppm
Sep 29 10:54:25 dist-test-slave-vqch krb5kdc[10707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679265, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:54:25.922049 10795 init.cc:377] Logged in from keytab as kudu/127.20.80.253@KRBTEST.COM (short username kudu)
I20260929 10:54:25.922487 10795 webserver.cc:460] Webserver started at http://127.20.80.253:34025/ using document root <none> and password file <none>
I20260929 10:54:25.922737 10795 fs_manager.cc:359] Metadata directory not provided
I20260929 10:54:25.922814 10795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:54:25.922966 10795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:54:25.924069 10795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-1/data/instance:
uuid: "c0d388dae4d2405f9059bdc907070e30"
format_stamp: "Formatted at 2026-09-29 10:54:25 on dist-test-slave-vqch"
server_key: "5c0fd06935bd7848b8fe656e8ab4a2da"
server_key_iv: "09e2fd62a1f5210eacb9b82ac8b380d5"
server_key_version: "encryptionkey@0"
I20260929 10:54:25.924456 10795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-1/wal/instance:
uuid: "c0d388dae4d2405f9059bdc907070e30"
format_stamp: "Formatted at 2026-09-29 10:54:25 on dist-test-slave-vqch"
server_key: "5c0fd06935bd7848b8fe656e8ab4a2da"
server_key_iv: "09e2fd62a1f5210eacb9b82ac8b380d5"
server_key_version: "encryptionkey@0"
I20260929 10:54:25.925951 10795 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 10:54:25.926751 10810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:25.926972 10795 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:54:25.927086 10795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-1/wal
uuid: "c0d388dae4d2405f9059bdc907070e30"
format_stamp: "Formatted at 2026-09-29 10:54:25 on dist-test-slave-vqch"
server_key: "5c0fd06935bd7848b8fe656e8ab4a2da"
server_key_iv: "09e2fd62a1f5210eacb9b82ac8b380d5"
server_key_version: "encryptionkey@0"
I20260929 10:54:25.927174 10795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:54:25.933895 10788 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44085 } attempt: 2
I20260929 10:54:25.934405 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:25.934256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:44085 (local address 127.0.0.1:49456)
0929 10:54:25.934314 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:25.934375 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.253:44085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260929 10:54:25.940557 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:25.940423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:44085 (local address 127.0.0.1:49460)
0929 10:54:25.940479 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:25.940509 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.253:44085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260929 10:54:25.940793 10788 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.80.253:44085: Network error: Client connection negotiation failed: client connection to 127.20.80.253:44085: connect: Connection refused (error 111)
I20260929 10:54:25.954303 10795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:54:25.956337 10795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:54:25.956499 10795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:54:25.960553 10795 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.253:44085
I20260929 10:54:25.960608 10863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.253:44085 every 8 connection(s)
I20260929 10:54:25.961019 10795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-1/data/info.pb
I20260929 10:54:25.962045 10864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:54:25.963303 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 10795
I20260929 10:54:25.963434 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-1/wal/instance
I20260929 10:54:25.963620 20803 external_mini_cluster.cc:1428] Setting key 7625fa431f97526292d44f44a09e88f0
I20260929 10:54:25.963708 10864 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.80.254" port: 41381 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 10:54:25 dist-test-slave-vqch krb5kdc[10707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679265, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.253@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.80.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.252@KRBTEST.COM" created.
I20260929 10:54:25.975143 10791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:25.964501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42144 (local address 127.20.80.254:41381)
0929 10:54:25.964561 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:25.964563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:25.964754 (+ 191us) server_negotiation.cc:407] Connection header received
0929 10:54:25.965541 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:25.965542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:25.965557 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:25.965609 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:25.966431 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:25.967256 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:25.968260 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:25.968643 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:25.968731 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:25.971518 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:25.971519 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:25.971520 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:25.971530 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:25.973452 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:25.974004 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:25.974005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:25.974006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:25.974110 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:25.974416 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:25.974417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:25.974418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:25.974556 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:25.974635 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:25.974935 (+ 300us) server_negotiation.cc:299] Negotiation successful
0929 10:54:25.975031 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 10:54:25.975373 10866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:25.964447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:41381 (local address 127.0.0.1:42144)
0929 10:54:25.964635 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:25.964671 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 10:54:25.965420 (+ 749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:54:25.965686 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:54:25.965690 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:54:25.966066 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:54:25.966334 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:25.966339 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:25.967364 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:25.967365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:25.968167 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:25.968169 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:25.968759 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:25.968760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:25.968891 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:25.969481 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:54:25.969489 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:54:25.970930 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:54:25.973572 (+ 2642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:25.973581 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:25.973584 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:25.973917 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:25.974201 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:25.974202 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:25.974202 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:25.974336 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:25.974641 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:54:25.974649 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:54:25.974802 (+ 153us) client_negotiation.cc:769] Sending connection context
0929 10:54:25.974913 (+ 111us) client_negotiation.cc:240] Negotiation successful
0929 10:54:25.975098 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":89,"threads_started":1}
I20260929 10:54:25.975955 10864 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44085 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 10:54:25 dist-test-slave-vqch krb5kdc[10707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679265, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.253@KRBTEST.COM for kudu/127.20.80.253@KRBTEST.COM
I20260929 10:54:25.983814 10866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:25.976229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:44085 (local address 127.0.0.1:49464)
0929 10:54:25.976282 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:25.976292 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:54:25.976329 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:54:25.976597 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:54:25.976598 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:54:25.976755 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:54:25.976803 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:25.976804 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:25.977783 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:25.977784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:25.978289 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:25.978290 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:25.978783 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:25.978784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:25.978877 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:25.979285 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:54:25.979291 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:54:25.980695 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:54:25.982659 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:25.982660 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:25.982660 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:25.982829 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:25.983080 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:25.983081 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:25.983081 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:25.983137 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:25.983438 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:54:25.983439 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:54:25.983477 (+ 38us) client_negotiation.cc:769] Sending connection context
0929 10:54:25.983541 (+ 64us) client_negotiation.cc:240] Negotiation successful
0929 10:54:25.983658 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":2}
I20260929 10:54:25.983825 10869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:25.976320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49464 (local address 127.20.80.253:44085)
0929 10:54:25.976465 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:25.976468 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:25.976485 (+ 17us) server_negotiation.cc:407] Connection header received
0929 10:54:25.976500 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:25.976501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:25.976511 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:25.976583 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:25.976891 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:25.977674 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:25.978390 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:25.978699 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:25.978760 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:25.980782 (+ 2022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:25.980806 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:25.980806 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:25.980812 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:25.982553 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:25.982916 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:25.982919 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:25.982920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:25.983004 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:25.983208 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:25.983209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:25.983209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:25.983367 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:25.983460 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:25.983570 (+ 110us) server_negotiation.cc:299] Negotiation successful
0929 10:54:25.983708 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":70,"server-negotiator.queue_time_us":122,"thread_start_us":98,"threads_started":1}
I20260929 10:54:25.984629 10864 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.80.252" port: 36567 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:54:25.985108 10866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:25.984929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:36567 (local address 127.0.0.1:60510)
0929 10:54:25.985028 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:25.985086 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.252:36567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260929 10:54:25.985281 10811 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.80.252:36567: connect: Connection refused (error 111)
I20260929 10:54:25.985739 10866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:25.985663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:36567 (local address 127.0.0.1:60516)
0929 10:54:25.985698 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:25.985723 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.252:36567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260929 10:54:25.985894 10864 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.80.252:36567: Network error: Client connection negotiation failed: client connection to 127.20.80.252:36567: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.252.keytab.
Entry for principal kudu/127.20.80.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.252.keytab.
I20260929 10:54:25.991504 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.252:36567
--webserver_interface=127.20.80.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.252.keytab
--principal=kudu/127.20.80.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:43635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:41381,127.20.80.253:44085,127.20.80.252:36567
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:54:26.031960 10788 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44085 } attempt: 3
I20260929 10:54:26.035135 10864 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.80.252" port: 36567 } attempt: 1
I20260929 10:54:26.035550 10866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:26.035423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:36567 (local address 127.0.0.1:60520)
0929 10:54:26.035482 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:26.035520 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.252:36567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260929 10:54:26.035954 10866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:26.035826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:36567 (local address 127.0.0.1:60522)
0929 10:54:26.035889 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:26.035925 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.252:36567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260929 10:54:26.036115 10864 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.80.252:36567: Network error: Client connection negotiation failed: client connection to 127.20.80.252:36567: connect: Connection refused (error 111)
Sep 29 10:54:26 dist-test-slave-vqch krb5kdc[10707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679265, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for kudu/127.20.80.253@KRBTEST.COM
I20260929 10:54:26.039462 10869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:26.032435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49476 (local address 127.20.80.253:44085)
0929 10:54:26.032505 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:26.032508 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:26.032519 (+ 11us) server_negotiation.cc:407] Connection header received
0929 10:54:26.032581 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:26.032582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:26.032595 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:26.032667 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:26.033010 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.033515 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.034257 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.034484 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.034563 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:26.036772 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:26.036773 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:26.036774 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:26.036781 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:26.038305 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:26.038663 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:26.038664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:26.038664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:26.038725 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:26.038932 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:26.038933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:26.038933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:26.039050 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:26.039120 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:26.039284 (+ 164us) server_negotiation.cc:299] Negotiation successful
0929 10:54:26.039369 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 10:54:26.039462 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:26.032364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:44085 (local address 127.0.0.1:49476)
0929 10:54:26.032423 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:26.032443 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 10:54:26.032516 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:54:26.032674 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:54:26.032675 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:54:26.032849 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:54:26.032919 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:26.032921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.033607 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.033608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:26.034164 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:26.034166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.034564 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.034566 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:26.034651 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:26.035176 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:54:26.035183 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:54:26.036679 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:54:26.038409 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:26.038410 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:26.038411 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:26.038588 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:26.038801 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:26.038802 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:26.038802 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:26.038868 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:26.039124 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:54:26.039125 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:54:26.039174 (+ 49us) client_negotiation.cc:769] Sending connection context
0929 10:54:26.039260 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 10:54:26.039369 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260929 10:54:26.040356 10788 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.80.252" port: 36567 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:54:26.040707 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:26.040614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:36567 (local address 127.0.0.1:60536)
0929 10:54:26.040659 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:26.040687 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.252:36567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260929 10:54:26.041074 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:26.040951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:36567 (local address 127.0.0.1:60546)
0929 10:54:26.041023 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:26.041047 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.252:36567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260929 10:54:26.041157 10788 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.80.252:36567: Network error: Client connection negotiation failed: client connection to 127.20.80.252:36567: connect: Connection refused (error 111)
I20260929 10:54:26.072304 10788 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.80.252" port: 36567 } attempt: 1
I20260929 10:54:26.072880 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:26.072732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:36567 (local address 127.0.0.1:60560)
0929 10:54:26.072798 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:26.072838 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.252:36567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260929 10:54:26.073277 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:26.073141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:36567 (local address 127.0.0.1:60576)
0929 10:54:26.073205 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:26.073246 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.252:36567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260929 10:54:26.073414 10788 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.80.252:36567: Network error: Client connection negotiation failed: client connection to 127.20.80.252:36567: connect: Connection refused (error 111)
W20260929 10:54:26.075757 10871 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 10:54:26.075949 10871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:54:26.075966 10871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:54:26.076014 10871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:54:26.077226 10871 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:54:26.077265 10871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:54:26.077279 10871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:54:26.077293 10871 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:54:26.077334 10871 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:54:26.078819 10871 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:43635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:41381,127.20.80.253:44085,127.20.80.252:36567
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.252.keytab
--principal=kudu/127.20.80.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.252:36567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.80.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:54:26.079071 10871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:54:26.079308 10871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:54:26.084550 10879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:26.084586 10876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:26.084595 10871 server_base.cc:1034] running on GCE node
W20260929 10:54:26.084604 10877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:26.085127 10871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:54:26.085479 10871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:54:26.086678 10871 hybrid_clock.cc:648] HybridClock initialized: now 1790679266086665 us; error 32 us; skew 500 ppm
Sep 29 10:54:26 dist-test-slave-vqch krb5kdc[10707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679266, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:54:26.089352 10871 init.cc:377] Logged in from keytab as kudu/127.20.80.252@KRBTEST.COM (short username kudu)
I20260929 10:54:26.089751 10871 webserver.cc:460] Webserver started at http://127.20.80.252:33929/ using document root <none> and password file <none>
I20260929 10:54:26.089952 10871 fs_manager.cc:359] Metadata directory not provided
I20260929 10:54:26.089993 10871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:54:26.090080 10871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:54:26.091120 10871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-2/data/instance:
uuid: "be6a3c2e840a41b49a4d5da0b006dd1c"
format_stamp: "Formatted at 2026-09-29 10:54:26 on dist-test-slave-vqch"
server_key: "ad5f2888c73157fad0616a2843154ddf"
server_key_iv: "be0a36dbc1cd288f7d5ea35fd180d171"
server_key_version: "encryptionkey@0"
I20260929 10:54:26.091439 10871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-2/wal/instance:
uuid: "be6a3c2e840a41b49a4d5da0b006dd1c"
format_stamp: "Formatted at 2026-09-29 10:54:26 on dist-test-slave-vqch"
server_key: "ad5f2888c73157fad0616a2843154ddf"
server_key_iv: "be0a36dbc1cd288f7d5ea35fd180d171"
server_key_version: "encryptionkey@0"
I20260929 10:54:26.092711 10871 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 10:54:26.093490 10886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:26.093655 10871 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:26.093732 10871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-2/wal
uuid: "be6a3c2e840a41b49a4d5da0b006dd1c"
format_stamp: "Formatted at 2026-09-29 10:54:26 on dist-test-slave-vqch"
server_key: "ad5f2888c73157fad0616a2843154ddf"
server_key_iv: "be0a36dbc1cd288f7d5ea35fd180d171"
server_key_version: "encryptionkey@0"
I20260929 10:54:26.093820 10871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:54:26.104274 10864 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.80.252" port: 36567 } attempt: 2
I20260929 10:54:26.104672 10866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:26.104547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:36567 (local address 127.0.0.1:60592)
0929 10:54:26.104606 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:26.104643 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.252:36567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260929 10:54:26.104665 10871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:54:26.105091 10866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:26.104919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:36567 (local address 127.0.0.1:60596)
0929 10:54:26.105026 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:26.105061 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.252:36567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260929 10:54:26.105254 10864 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.80.252:36567: Network error: Client connection negotiation failed: client connection to 127.20.80.252:36567: connect: Connection refused (error 111)
I20260929 10:54:26.106743 10871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:54:26.106906 10871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:54:26.110716 10871 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.252:36567
I20260929 10:54:26.110762 10938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.252:36567 every 8 connection(s)
I20260929 10:54:26.111111 10871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-2/data/info.pb
I20260929 10:54:26.112104 10939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:54:26.113708 10939 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.80.254" port: 41381 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:54:26.116138 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 10871
I20260929 10:54:26.116230 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790678996126930-20803-0/minicluster-data/master-2/wal/instance
I20260929 10:54:26.116417 20803 external_mini_cluster.cc:1428] Setting key 877502a2ed1b7dd0fa4b4002693f67f5
Sep 29 10:54:26 dist-test-slave-vqch krb5kdc[10707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679266, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.252@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
Sep 29 10:54:26 dist-test-slave-vqch krb5kdc[10707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.252@KRBTEST.COM
I20260929 10:54:26.123476 10791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:26.114360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42154 (local address 127.20.80.254:41381)
0929 10:54:26.114415 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:26.114417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:26.114606 (+ 189us) server_negotiation.cc:407] Connection header received
0929 10:54:26.115051 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:26.115052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:26.115062 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:26.115128 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:26.115719 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.116222 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.117568 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.117773 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.117848 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:26.120047 (+ 2199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:26.120048 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:26.120049 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:26.120056 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:26.121995 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:26.122463 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:26.122465 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:26.122465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:26.122527 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:26.122792 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:26.122793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:26.122793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:26.122936 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:26.122987 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:26.123325 (+ 338us) server_negotiation.cc:299] Negotiation successful
0929 10:54:26.123405 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260929 10:54:26.123721 10941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:26.114316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:41381 (local address 127.0.0.1:42154)
0929 10:54:26.114487 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:26.114529 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 10:54:26.114967 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:54:26.115206 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:54:26.115210 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:54:26.115442 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:54:26.115619 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:26.115623 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.116311 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.116312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:26.117490 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:26.117492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.117886 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.117888 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:26.118014 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:26.118502 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:54:26.118510 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:54:26.119959 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:54:26.122101 (+ 2142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:26.122108 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:26.122110 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:26.122393 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:26.122599 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:26.122600 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:26.122600 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:26.122742 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:26.123002 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:54:26.123007 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:54:26.123159 (+ 152us) client_negotiation.cc:769] Sending connection context
0929 10:54:26.123287 (+ 128us) client_negotiation.cc:240] Negotiation successful
0929 10:54:26.123456 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":85,"threads_started":1}
I20260929 10:54:26.124379 10939 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44085 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:54:26.125802 10943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:26.116804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60608 (local address 127.20.80.252:36567)
0929 10:54:26.116945 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:26.116949 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:26.117117 (+ 168us) server_negotiation.cc:407] Connection header received
0929 10:54:26.117144 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:26.117146 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:26.117167 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:26.117247 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:26.117801 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.118641 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.119688 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.119993 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.120088 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:26.122859 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:26.122864 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:26.122865 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:26.122870 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:26.124576 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:26.124928 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:26.124931 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:26.124931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:26.125013 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:26.125210 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:26.125211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:26.125211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:26.125355 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:26.125516 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:26.125610 (+ 94us) server_negotiation.cc:299] Negotiation successful
0929 10:54:26.125699 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":73,"threads_started":1}
Sep 29 10:54:26 dist-test-slave-vqch krb5kdc[10707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679266, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.252@KRBTEST.COM for kudu/127.20.80.253@KRBTEST.COM
Sep 29 10:54:26 dist-test-slave-vqch krb5kdc[10707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:54:26.131420 10869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:26.124694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49478 (local address 127.20.80.253:44085)
0929 10:54:26.124746 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:26.124748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:26.124756 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:54:26.124784 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:26.124785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:26.124793 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:26.124853 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:26.125141 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.125585 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.126471 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.126803 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.126870 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:26.128722 (+ 1852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:26.128723 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:26.128723 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:26.128731 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:26.130266 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:26.130621 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:26.130623 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:26.130623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:26.130680 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:26.130866 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:26.130867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:26.130867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:26.130997 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:26.131073 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:26.131298 (+ 225us) server_negotiation.cc:299] Negotiation successful
0929 10:54:26.131358 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 10:54:26.131479 10941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:26.124609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:44085 (local address 127.0.0.1:49478)
0929 10:54:26.124657 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:26.124666 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:54:26.124700 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:54:26.124856 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:54:26.124856 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:54:26.125034 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:54:26.125075 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:26.125076 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.125667 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.125668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:26.126386 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:26.126388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.126876 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.126877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:26.126924 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:26.127303 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:54:26.127307 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:54:26.128639 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:54:26.130348 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:26.130349 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:26.130349 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:26.130552 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:26.130754 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:26.130754 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:26.130755 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:26.130804 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:26.131133 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:54:26.131134 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:54:26.131183 (+ 49us) client_negotiation.cc:769] Sending connection context
0929 10:54:26.131268 (+ 85us) client_negotiation.cc:240] Negotiation successful
0929 10:54:26.131364 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260929 10:54:26.131974 10939 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.80.252" port: 36567 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:54:26.134343 10791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:26.126552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42170 (local address 127.20.80.254:41381)
0929 10:54:26.126594 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:26.126596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:26.126648 (+ 52us) server_negotiation.cc:407] Connection header received
0929 10:54:26.126717 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:26.126718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:26.126725 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:26.126763 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:26.127368 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.128010 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.128836 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.129066 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.129138 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:26.131606 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:26.131607 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:26.131608 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:26.131615 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:26.133222 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:26.133560 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:26.133561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:26.133561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:26.133620 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:26.133818 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:26.133819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:26.133819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:26.133933 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:26.134109 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:26.134156 (+ 47us) server_negotiation.cc:299] Negotiation successful
0929 10:54:26.134234 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Sep 29 10:54:26 dist-test-slave-vqch krb5kdc[10707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679266, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.252@KRBTEST.COM for kudu/127.20.80.252@KRBTEST.COM
I20260929 10:54:26.138537 10941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:26.132235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:36567 (local address 127.0.0.1:60622)
0929 10:54:26.132280 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:26.132289 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:54:26.132333 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:54:26.132440 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:54:26.132441 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:54:26.132571 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:54:26.132602 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:26.132603 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.133272 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.133273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:26.133734 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:26.133735 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.134122 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.134123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:26.134170 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:26.134572 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:54:26.134577 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:54:26.135910 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:54:26.137526 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:26.137526 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:26.137527 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:26.137680 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:26.137882 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:26.137883 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:26.137884 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:26.137938 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:26.138177 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:54:26.138178 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:54:26.138217 (+ 39us) client_negotiation.cc:769] Sending connection context
0929 10:54:26.138296 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 10:54:26.138414 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260929 10:54:26.138588 10943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:26.132305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60622 (local address 127.20.80.252:36567)
0929 10:54:26.132357 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:26.132359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:26.132368 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:54:26.132385 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:26.132386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:26.132394 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:26.132439 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:26.132671 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.133187 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.133826 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.134036 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.134122 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:26.135999 (+ 1877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:26.136000 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:26.136000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:26.136006 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:26.137450 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:26.137757 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:26.137758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:26.137758 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:26.137809 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:26.138008 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:26.138008 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:26.138009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:26.138112 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:26.138184 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:26.138319 (+ 135us) server_negotiation.cc:299] Negotiation successful
0929 10:54:26.138461 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":79,"server-negotiator.queue_time_us":36}
Sep 29 10:54:26 dist-test-slave-vqch krb5kdc[10707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.253@KRBTEST.COM
I20260929 10:54:26.140308 10939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be6a3c2e840a41b49a4d5da0b006dd1c: Bootstrap starting.
I20260929 10:54:26.141264 10939 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be6a3c2e840a41b49a4d5da0b006dd1c: Neither blocks nor log segments found. Creating new log.
I20260929 10:54:26.141557 10939 log.cc:824] T 00000000000000000000000000000000 P be6a3c2e840a41b49a4d5da0b006dd1c: Log is configured to *not* fsync() on all Append() calls
I20260929 10:54:26.142110 10869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:26.134964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49480 (local address 127.20.80.253:44085)
0929 10:54:26.134999 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:26.135001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:26.135008 (+ 7us) server_negotiation.cc:407] Connection header received
0929 10:54:26.135027 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:26.135028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:26.135037 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:26.135089 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:26.135731 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.136274 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.136906 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.137179 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.137263 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:26.139493 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:26.139494 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:26.139495 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:26.139501 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:26.140941 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:26.141346 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:26.141348 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:26.141348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:26.141402 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:26.141604 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:26.141605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:26.141606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:26.141726 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:26.141908 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:26.141944 (+ 36us) server_negotiation.cc:299] Negotiation successful
0929 10:54:26.142013 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 10:54:26.142333 10939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be6a3c2e840a41b49a4d5da0b006dd1c: No bootstrap required, opened a new log
I20260929 10:54:26.143690 20803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:54:26.144249 10939 raft_consensus.cc:348] T 00000000000000000000000000000000 P be6a3c2e840a41b49a4d5da0b006dd1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0339829159b64354a6bf01b5324e1f79" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 41381 } } peers { permanent_uuid: "c0d388dae4d2405f9059bdc907070e30" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44085 } } peers { permanent_uuid: "be6a3c2e840a41b49a4d5da0b006dd1c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 36567 } }
I20260929 10:54:26.144395 10939 raft_consensus.cc:729] T 00000000000000000000000000000000 P be6a3c2e840a41b49a4d5da0b006dd1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be6a3c2e840a41b49a4d5da0b006dd1c, State: Initialized, Role: FOLLOWER
I20260929 10:54:26.144498 10939 consensus_queue.cc:260] T 00000000000000000000000000000000 P be6a3c2e840a41b49a4d5da0b006dd1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0339829159b64354a6bf01b5324e1f79" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 41381 } } peers { permanent_uuid: "c0d388dae4d2405f9059bdc907070e30" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44085 } } peers { permanent_uuid: "be6a3c2e840a41b49a4d5da0b006dd1c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 36567 } }
I20260929 10:54:26.144790 10945 sys_catalog.cc:455] T 00000000000000000000000000000000 P be6a3c2e840a41b49a4d5da0b006dd1c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0339829159b64354a6bf01b5324e1f79" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 41381 } } peers { permanent_uuid: "c0d388dae4d2405f9059bdc907070e30" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44085 } } peers { permanent_uuid: "be6a3c2e840a41b49a4d5da0b006dd1c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 36567 } } }
I20260929 10:54:26.144848 10939 sys_catalog.cc:564] T 00000000000000000000000000000000 P be6a3c2e840a41b49a4d5da0b006dd1c [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:54:26.144896 10945 sys_catalog.cc:458] T 00000000000000000000000000000000 P be6a3c2e840a41b49a4d5da0b006dd1c [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:54:26.144627 10762 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "0339829159b64354a6bf01b5324e1f79"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42170
W20260929 10:54:26.146907 10956 catalog_manager.cc:1558] T 00000000000000000000000000000000 P be6a3c2e840a41b49a4d5da0b006dd1c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:54:26.146962 10956 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:54:26.148530 10788 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.80.252" port: 36567 } attempt: 2
I20260929 10:54:26.147199 10871 master_runner.cc:186] Error getting master registration for 127.20.80.254:41381: OK, instance_id { permanent_uuid: "0339829159b64354a6bf01b5324e1f79" instance_seqno: 1790679265798559 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 10:54:26.150242 10871 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Sep 29 10:54:26 dist-test-slave-vqch krb5kdc[10707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679265, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for kudu/127.20.80.252@KRBTEST.COM
I20260929 10:54:26.156383 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:26.148775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:36567 (local address 127.0.0.1:60638)
0929 10:54:26.148822 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:26.148834 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:54:26.148885 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:54:26.149075 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:54:26.149077 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:54:26.149267 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:54:26.149314 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:26.149316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.150334 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.150335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:26.150856 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:26.150857 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.151344 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.151345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:26.151407 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:26.151896 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:54:26.151901 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:54:26.153400 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:54:26.155209 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:26.155210 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:26.155210 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:26.155453 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:26.155678 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:26.155678 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:26.155679 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:26.155743 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:26.156055 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:54:26.156056 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:54:26.156103 (+ 47us) client_negotiation.cc:769] Sending connection context
0929 10:54:26.156171 (+ 68us) client_negotiation.cc:240] Negotiation successful
0929 10:54:26.156277 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260929 10:54:26.156438 10943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:26.148822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60638 (local address 127.20.80.252:36567)
0929 10:54:26.148887 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:26.148889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:26.148899 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:54:26.148969 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:26.148970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:26.149003 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:26.149059 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:26.149400 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.150149 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.150940 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.151245 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.151323 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:26.153482 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:26.153484 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:26.153484 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:26.153492 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:26.155120 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:26.155533 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:26.155534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:26.155534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:26.155598 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:26.155808 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:26.155809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:26.155809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:26.155986 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:26.156058 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:26.156217 (+ 159us) server_negotiation.cc:299] Negotiation successful
0929 10:54:26.156308 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260929 10:54:26.158089 10788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0339829159b64354a6bf01b5324e1f79: Bootstrap starting.
I20260929 10:54:26.158720 10788 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0339829159b64354a6bf01b5324e1f79: Neither blocks nor log segments found. Creating new log.
I20260929 10:54:26.159009 10788 log.cc:824] T 00000000000000000000000000000000 P 0339829159b64354a6bf01b5324e1f79: Log is configured to *not* fsync() on all Append() calls
I20260929 10:54:26.159720 10788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0339829159b64354a6bf01b5324e1f79: No bootstrap required, opened a new log
I20260929 10:54:26.160710 10788 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0339829159b64354a6bf01b5324e1f79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0339829159b64354a6bf01b5324e1f79" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 41381 } } peers { permanent_uuid: "c0d388dae4d2405f9059bdc907070e30" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44085 } } peers { permanent_uuid: "be6a3c2e840a41b49a4d5da0b006dd1c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 36567 } }
I20260929 10:54:26.160841 10788 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0339829159b64354a6bf01b5324e1f79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0339829159b64354a6bf01b5324e1f79, State: Initialized, Role: FOLLOWER
I20260929 10:54:26.160955 10788 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0339829159b64354a6bf01b5324e1f79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0339829159b64354a6bf01b5324e1f79" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 41381 } } peers { permanent_uuid: "c0d388dae4d2405f9059bdc907070e30" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44085 } } peers { permanent_uuid: "be6a3c2e840a41b49a4d5da0b006dd1c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 36567 } }
I20260929 10:54:26.161262 10959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0339829159b64354a6bf01b5324e1f79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0339829159b64354a6bf01b5324e1f79" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 41381 } } peers { permanent_uuid: "c0d388dae4d2405f9059bdc907070e30" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44085 } } peers { permanent_uuid: "be6a3c2e840a41b49a4d5da0b006dd1c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 36567 } } }
I20260929 10:54:26.161329 10788 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0339829159b64354a6bf01b5324e1f79 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:54:26.161339 10959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0339829159b64354a6bf01b5324e1f79 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:54:26.161468 10762 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0339829159b64354a6bf01b5324e1f79 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260929 10:54:26.161542 10762 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0339829159b64354a6bf01b5324e1f79 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:54:26.162324 10762 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0339829159b64354a6bf01b5324e1f79 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0339829159b64354a6bf01b5324e1f79" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 41381 } } peers { permanent_uuid: "c0d388dae4d2405f9059bdc907070e30" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44085 } } peers { permanent_uuid: "be6a3c2e840a41b49a4d5da0b006dd1c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 36567 } }
W20260929 10:54:26.163151 10970 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0339829159b64354a6bf01b5324e1f79: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:54:26.163205 10970 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:54:26.162560 10762 leader_election.cc:288] T 00000000000000000000000000000000 P 0339829159b64354a6bf01b5324e1f79 [CANDIDATE]: Term 1 election: Requested vote from peers c0d388dae4d2405f9059bdc907070e30 (127.20.80.253:44085), be6a3c2e840a41b49a4d5da0b006dd1c (127.20.80.252:36567)
I20260929 10:54:26.163951 10913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0339829159b64354a6bf01b5324e1f79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "be6a3c2e840a41b49a4d5da0b006dd1c"
I20260929 10:54:26.164083 10913 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be6a3c2e840a41b49a4d5da0b006dd1c [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:54:26.165759 10720 master_runner.cc:186] Error getting master registration for 127.20.80.253:44085: OK, instance_id { permanent_uuid: "c0d388dae4d2405f9059bdc907070e30" instance_seqno: 1790679265959413 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 10:54:26.168020 10720 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 10:54:26.168365 10913 raft_consensus.cc:2454] T 00000000000000000000000000000000 P be6a3c2e840a41b49a4d5da0b006dd1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0339829159b64354a6bf01b5324e1f79 in term 1.
I20260929 10:54:26.168599 10736 leader_election.cc:302] T 00000000000000000000000000000000 P 0339829159b64354a6bf01b5324e1f79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0339829159b64354a6bf01b5324e1f79, be6a3c2e840a41b49a4d5da0b006dd1c; no voters:
I20260929 10:54:26.168715 10959 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0339829159b64354a6bf01b5324e1f79 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:54:26.168857 10959 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0339829159b64354a6bf01b5324e1f79 [term 1 LEADER]: Becoming Leader. State: Replica: 0339829159b64354a6bf01b5324e1f79, State: Running, Role: LEADER
I20260929 10:54:26.168933 10959 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0339829159b64354a6bf01b5324e1f79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0339829159b64354a6bf01b5324e1f79" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 41381 } } peers { permanent_uuid: "c0d388dae4d2405f9059bdc907070e30" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44085 } } peers { permanent_uuid: "be6a3c2e840a41b49a4d5da0b006dd1c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 36567 } }
I20260929 10:54:26.169355 10959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0339829159b64354a6bf01b5324e1f79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0339829159b64354a6bf01b5324e1f79. Latest consensus state: current_term: 1 leader_uuid: "0339829159b64354a6bf01b5324e1f79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0339829159b64354a6bf01b5324e1f79" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 41381 } } peers { permanent_uuid: "c0d388dae4d2405f9059bdc907070e30" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44085 } } peers { permanent_uuid: "be6a3c2e840a41b49a4d5da0b006dd1c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 36567 } } }
I20260929 10:54:26.169461 10959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0339829159b64354a6bf01b5324e1f79 [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:26.175489 10869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:26.166490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49492 (local address 127.20.80.253:44085)
0929 10:54:26.166531 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:26.166534 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:26.167152 (+ 618us) server_negotiation.cc:407] Connection header received
0929 10:54:26.167241 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:26.167242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:26.167253 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:26.167282 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:26.168170 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.168930 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.170564 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.170896 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.170940 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:26.172544 (+ 1604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:26.172546 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:26.172546 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:26.172553 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:26.174278 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:26.174672 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:26.174673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:26.174674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:26.174729 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:26.174922 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:26.174923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:26.174923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:26.175042 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:26.175109 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:26.175311 (+ 202us) server_negotiation.cc:299] Negotiation successful
0929 10:54:26.175389 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 10:54:26.177161 10981 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:54:26.177398 10981 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:54:26.179492 10913 raft_consensus.cc:1261] T 00000000000000000000000000000000 P be6a3c2e840a41b49a4d5da0b006dd1c [term 1 FOLLOWER]: Refusing update from remote peer 0339829159b64354a6bf01b5324e1f79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:54:26.179878 10959 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0339829159b64354a6bf01b5324e1f79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be6a3c2e840a41b49a4d5da0b006dd1c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 36567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:54:26.181769 10945 sys_catalog.cc:455] T 00000000000000000000000000000000 P be6a3c2e840a41b49a4d5da0b006dd1c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0339829159b64354a6bf01b5324e1f79. Latest consensus state: current_term: 1 leader_uuid: "0339829159b64354a6bf01b5324e1f79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0339829159b64354a6bf01b5324e1f79" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 41381 } } peers { permanent_uuid: "c0d388dae4d2405f9059bdc907070e30" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44085 } } peers { permanent_uuid: "be6a3c2e840a41b49a4d5da0b006dd1c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 36567 } } }
I20260929 10:54:26.181857 10945 sys_catalog.cc:458] T 00000000000000000000000000000000 P be6a3c2e840a41b49a4d5da0b006dd1c [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:54:26.183892 10791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:26.168135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42174 (local address 127.20.80.254:41381)
0929 10:54:26.168190 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:26.168192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:26.168200 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:54:26.168219 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:26.168220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:26.168235 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:26.168290 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:26.170839 (+ 2549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.171594 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.173370 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.173669 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.173710 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:26.178695 (+ 4985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:26.178697 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:26.178697 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:26.178705 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:26.180502 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:26.182184 (+ 1682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:26.182186 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:26.182186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:26.182266 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:26.182977 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:26.182978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:26.182979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:26.183123 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:26.183217 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:26.183722 (+ 505us) server_negotiation.cc:299] Negotiation successful
0929 10:54:26.183797 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 10:54:26.186472 10959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0339829159b64354a6bf01b5324e1f79 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0339829159b64354a6bf01b5324e1f79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0339829159b64354a6bf01b5324e1f79" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 41381 } } peers { permanent_uuid: "c0d388dae4d2405f9059bdc907070e30" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44085 } } peers { permanent_uuid: "be6a3c2e840a41b49a4d5da0b006dd1c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 36567 } } }
I20260929 10:54:26.186554 10959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0339829159b64354a6bf01b5324e1f79 [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:26.187093 10945 sys_catalog.cc:455] T 00000000000000000000000000000000 P be6a3c2e840a41b49a4d5da0b006dd1c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0339829159b64354a6bf01b5324e1f79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0339829159b64354a6bf01b5324e1f79" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 41381 } } peers { permanent_uuid: "c0d388dae4d2405f9059bdc907070e30" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44085 } } peers { permanent_uuid: "be6a3c2e840a41b49a4d5da0b006dd1c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 36567 } } }
I20260929 10:54:26.187175 10945 sys_catalog.cc:458] T 00000000000000000000000000000000 P be6a3c2e840a41b49a4d5da0b006dd1c [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:54:26.187464 10982 mvcc.cc:205] Tried to move back new op lower bound from 7334622274268676096 to 7334622274228658176. Current Snapshot: MvccSnapshot[applied={T|T < 7334622274268676096}]
I20260929 10:54:26.188913 10943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:26.166797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60650 (local address 127.20.80.252:36567)
0929 10:54:26.166831 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:26.166832 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:26.170317 (+ 3485us) server_negotiation.cc:407] Connection header received
0929 10:54:26.170409 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:26.170410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:26.170419 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:26.170446 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:26.175457 (+ 5011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.176169 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.181214 (+ 5045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.181535 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.181575 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:26.185600 (+ 4025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:26.185602 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:26.185602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:26.185609 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:26.187365 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:26.187917 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:26.187918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:26.187918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:26.187982 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:26.188266 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:26.188267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:26.188267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:26.188392 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:26.188470 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:26.188736 (+ 266us) server_negotiation.cc:299] Negotiation successful
0929 10:54:26.188825 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 10:54:26.189033 10983 mvcc.cc:205] Tried to move back new op lower bound from 7334622274268676096 to 7334622274228658176. Current Snapshot: MvccSnapshot[applied={T|T < 7334622274268676096}]
I20260929 10:54:26.193399 10981 catalog_manager.cc:1347] Generated new cluster ID: 563d59b03b2142c8b1fe9985461215e6
I20260929 10:54:26.193451 10981 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:54:26.196393 10864 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.80.252" port: 36567 } attempt: 3
Sep 29 10:54:26 dist-test-slave-vqch krb5kdc[10707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679265, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.253@KRBTEST.COM for kudu/127.20.80.252@KRBTEST.COM
I20260929 10:54:26.204903 10866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:26.196710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:36567 (local address 127.0.0.1:60660)
0929 10:54:26.196787 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:26.196797 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:54:26.196832 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:54:26.197543 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:54:26.197544 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:54:26.197721 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:54:26.197767 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:26.197769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.198648 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.198649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:26.199436 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:26.199438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.199962 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.199963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:26.200017 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:26.200452 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:54:26.200457 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:54:26.201977 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:54:26.203755 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:26.203756 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:26.203757 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:26.203964 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:26.204201 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:26.204202 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:26.204202 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:26.204271 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:26.204577 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:54:26.204578 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:54:26.204640 (+ 62us) client_negotiation.cc:769] Sending connection context
0929 10:54:26.204701 (+ 61us) client_negotiation.cc:240] Negotiation successful
0929 10:54:26.204796 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260929 10:54:26.205593 10943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:26.196910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60660 (local address 127.20.80.252:36567)
0929 10:54:26.197411 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:26.197413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:26.197420 (+ 7us) server_negotiation.cc:407] Connection header received
0929 10:54:26.197438 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:26.197438 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:26.197447 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:26.197498 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:26.197849 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.198544 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.199543 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:26.199853 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:26.199911 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:26.202057 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:26.202059 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:26.202059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:26.202066 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:26.203665 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:26.204055 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:26.204056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:26.204057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:26.204121 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:26.204346 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:26.204347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:26.204347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:26.204483 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:26.204540 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:26.205427 (+ 887us) server_negotiation.cc:299] Negotiation successful
0929 10:54:26.205503 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482}
I20260929 10:54:26.207319 10864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0d388dae4d2405f9059bdc907070e30: Bootstrap starting.
I20260929 10:54:26.208068 10864 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0d388dae4d2405f9059bdc907070e30: Neither blocks nor log segments found. Creating new log.
I20260929 10:54:26.208359 10864 log.cc:824] T 00000000000000000000000000000000 P c0d388dae4d2405f9059bdc907070e30: Log is configured to *not* fsync() on all Append() calls
I20260929 10:54:26.217681 10864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0d388dae4d2405f9059bdc907070e30: No bootstrap required, opened a new log
I20260929 10:54:26.219898 10864 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0d388dae4d2405f9059bdc907070e30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0339829159b64354a6bf01b5324e1f79" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 41381 } } peers { permanent_uuid: "c0d388dae4d2405f9059bdc907070e30" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44085 } } peers { permanent_uuid: "be6a3c2e840a41b49a4d5da0b006dd1c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 36567 } }
I20260929 10:54:26.220050 10864 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0d388dae4d2405f9059bdc907070e30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0d388dae4d2405f9059bdc907070e30, State: Initialized, Role: FOLLOWER
I20260929 10:54:26.220144 10864 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0d388dae4d2405f9059bdc907070e30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0339829159b64354a6bf01b5324e1f79" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 41381 } } peers { permanent_uuid: "c0d388dae4d2405f9059bdc907070e30" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44085 } } peers { permanent_uuid: "be6a3c2e840a41b49a4d5da0b006dd1c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 36567 } }
I20260929 10:54:26.220506 10864 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0d388dae4d2405f9059bdc907070e30 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:54:26.220710 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0339829159b64354a6bf01b5324e1f79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c0d388dae4d2405f9059bdc907070e30"
I20260929 10:54:26.221139 10837 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0d388dae4d2405f9059bdc907070e30 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:54:26.222074 10837 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c0d388dae4d2405f9059bdc907070e30 [term 1 FOLLOWER]: Refusing update from remote peer 0339829159b64354a6bf01b5324e1f79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:54:26.222664 10979 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0339829159b64354a6bf01b5324e1f79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0d388dae4d2405f9059bdc907070e30" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:54:26.223541 10981 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:54:26.224032 10838 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c0d388dae4d2405f9059bdc907070e30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0339829159b64354a6bf01b5324e1f79 in term 1.
I20260929 10:54:26.224277 10981 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:54:26.225661 10989 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0d388dae4d2405f9059bdc907070e30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0339829159b64354a6bf01b5324e1f79" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 41381 } } peers { permanent_uuid: "c0d388dae4d2405f9059bdc907070e30" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44085 } } peers { permanent_uuid: "be6a3c2e840a41b49a4d5da0b006dd1c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 36567 } } }
I20260929 10:54:26.225821 10989 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0d388dae4d2405f9059bdc907070e30 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:54:26.225919 10989 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0d388dae4d2405f9059bdc907070e30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0339829159b64354a6bf01b5324e1f79. Latest consensus state: current_term: 1 leader_uuid: "0339829159b64354a6bf01b5324e1f79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0339829159b64354a6bf01b5324e1f79" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 41381 } } peers { permanent_uuid: "c0d388dae4d2405f9059bdc907070e30" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44085 } } peers { permanent_uuid: "be6a3c2e840a41b49a4d5da0b006dd1c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 36567 } } }
I20260929 10:54:26.226023 10989 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0d388dae4d2405f9059bdc907070e30 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:54:26.226091 10989 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0d388dae4d2405f9059bdc907070e30 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0339829159b64354a6bf01b5324e1f79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0339829159b64354a6bf01b5324e1f79" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 41381 } } peers { permanent_uuid: "c0d388dae4d2405f9059bdc907070e30" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44085 } } peers { permanent_uuid: "be6a3c2e840a41b49a4d5da0b006dd1c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 36567 } } }
I20260929 10:54:26.226179 10989 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0d388dae4d2405f9059bdc907070e30 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:54:26.227075 10986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0339829159b64354a6bf01b5324e1f79 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0339829159b64354a6bf01b5324e1f79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0339829159b64354a6bf01b5324e1f79" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 41381 } } peers { permanent_uuid: "c0d388dae4d2405f9059bdc907070e30" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 44085 } } peers { permanent_uuid: "be6a3c2e840a41b49a4d5da0b006dd1c" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 36567 } } }
I20260929 10:54:26.227173 10986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0339829159b64354a6bf01b5324e1f79 [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:26.228080 11001 mvcc.cc:205] Tried to move back new op lower bound from 7334622274430664704 to 7334622274228658176. Current Snapshot: MvccSnapshot[applied={T|T < 7334622274268676096}]
I20260929 10:54:26.233827 11006 catalog_manager.cc:1259] Loaded cluster ID: 563d59b03b2142c8b1fe9985461215e6
I20260929 10:54:26.233927 11006 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c0d388dae4d2405f9059bdc907070e30: loading cluster ID for follower catalog manager: success
I20260929 10:54:26.235150 11006 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c0d388dae4d2405f9059bdc907070e30: acquiring CA information for follower catalog manager: success
W20260929 10:54:26.235342 11006 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260929 10:54:26.239061 10981 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0339829159b64354a6bf01b5324e1f79: Generated new TSK 0
I20260929 10:54:26.239238 10981 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 10:54:26.264045 10791 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:54:26.262348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42178 (local address 127.20.80.254:41381)
0929 10:54:26.262394 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:26.262396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:26.263351 (+ 955us) server_negotiation.cc:407] Connection header received
0929 10:54:26.263460 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:26.263462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:26.263473 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:26.263505 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:26.263972 (+ 467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42178: BlockingRecv error: recv got EOF from 127.0.0.1:42178 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260929 10:54:26.265019 10869 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:54:26.262733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49508 (local address 127.20.80.253:44085)
0929 10:54:26.262776 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:26.262778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:26.264465 (+ 1687us) server_negotiation.cc:407] Connection header received
0929 10:54:26.264568 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:26.264569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:26.264580 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:26.264610 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:26.264927 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49508: BlockingRecv error: recv got EOF from 127.0.0.1:49508 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260929 10:54:26.265899 10943 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:54:26.263080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60668 (local address 127.20.80.252:36567)
0929 10:54:26.263120 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:26.263122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:26.265383 (+ 2261us) server_negotiation.cc:407] Connection header received
0929 10:54:26.265481 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:26.265482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:26.265493 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:26.265524 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:26.265836 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60668: BlockingRecv error: recv got EOF from 127.0.0.1:60668 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260929 10:54:26.270272 10869 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:54:26.269491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49518 (local address 127.20.80.253:44085)
0929 10:54:26.269532 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:26.269533 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:26.269734 (+ 201us) server_negotiation.cc:407] Connection header received
0929 10:54:26.269836 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:26.269836 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:26.269857 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:26.269886 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:26.270218 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49518: BlockingRecv error: recv got EOF from 127.0.0.1:49518 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260929 10:54:26.273413 10869 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:54:26.272621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49528 (local address 127.20.80.253:44085)
0929 10:54:26.272653 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:26.272655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:26.272880 (+ 225us) server_negotiation.cc:407] Connection header received
0929 10:54:26.273000 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:26.273001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:26.273012 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:26.273041 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:26.273372 (+ 331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49528: BlockingRecv error: recv got EOF from 127.0.0.1:49528 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260929 10:54:26.277431 10869 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:54:26.276590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49540 (local address 127.20.80.253:44085)
0929 10:54:26.276622 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:26.276624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:26.276842 (+ 218us) server_negotiation.cc:407] Connection header received
0929 10:54:26.276940 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:26.276941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:26.276950 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:26.277054 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:26.277390 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49540: BlockingRecv error: recv got EOF from 127.0.0.1:49540 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260929 10:54:26.283588 10869 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:54:26.282709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49552 (local address 127.20.80.253:44085)
0929 10:54:26.282747 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:26.282749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:26.282996 (+ 247us) server_negotiation.cc:407] Connection header received
0929 10:54:26.283108 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:26.283109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:26.283123 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:26.283158 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:26.283536 (+ 378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49552: BlockingRecv error: recv got EOF from 127.0.0.1:49552 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260929 10:54:26.294139 10869 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:54:26.293170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49558 (local address 127.20.80.253:44085)
0929 10:54:26.293215 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:26.293217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:26.293518 (+ 301us) server_negotiation.cc:407] Connection header received
0929 10:54:26.293625 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:26.293627 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:26.293639 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:26.293670 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:26.294068 (+ 398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49558: BlockingRecv error: recv got EOF from 127.0.0.1:49558 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260929 10:54:26.312779 10869 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:54:26.311772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49574 (local address 127.20.80.253:44085)
0929 10:54:26.311815 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:26.311817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:26.312107 (+ 290us) server_negotiation.cc:407] Connection header received
0929 10:54:26.312234 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:26.312235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:26.312248 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:26.312282 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:26.312715 (+ 433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49574: BlockingRecv error: recv got EOF from 127.0.0.1:49574 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260929 10:54:26.347115 10869 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:54:26.346389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49588 (local address 127.20.80.253:44085)
0929 10:54:26.346444 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:26.346446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:26.346559 (+ 113us) server_negotiation.cc:407] Connection header received
0929 10:54:26.346617 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:26.346618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:26.346631 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:26.346709 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:26.347040 (+ 331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49588: BlockingRecv error: recv got EOF from 127.0.0.1:49588 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260929 10:54:26.413264 10869 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:54:26.412414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49598 (local address 127.20.80.253:44085)
0929 10:54:26.412464 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:26.412467 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:26.412608 (+ 141us) server_negotiation.cc:407] Connection header received
0929 10:54:26.412718 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:26.412719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:26.412730 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:26.412793 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:26.413194 (+ 401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49598: BlockingRecv error: recv got EOF from 127.0.0.1:49598 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260929 10:54:26.543260 10869 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:54:26.542510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49600 (local address 127.20.80.253:44085)
0929 10:54:26.542598 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:26.542600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:26.542688 (+ 88us) server_negotiation.cc:407] Connection header received
0929 10:54:26.542740 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:26.542741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:26.542752 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:26.542820 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:26.543188 (+ 368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49600: BlockingRecv error: recv got EOF from 127.0.0.1:49600 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260929 10:54:26.801317 10869 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:54:26.800645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49604 (local address 127.20.80.253:44085)
0929 10:54:26.800734 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:26.800736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:26.800745 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:54:26.800775 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:26.800776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:26.800787 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:26.800860 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:26.801242 (+ 382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49604: BlockingRecv error: recv got EOF from 127.0.0.1:49604 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 10:54:27.147518 10956 catalog_manager.cc:1259] Loaded cluster ID: 563d59b03b2142c8b1fe9985461215e6
I20260929 10:54:27.147600 10956 catalog_manager.cc:1552] T 00000000000000000000000000000000 P be6a3c2e840a41b49a4d5da0b006dd1c: loading cluster ID for follower catalog manager: success
I20260929 10:54:27.148433 10956 catalog_manager.cc:1574] T 00000000000000000000000000000000 P be6a3c2e840a41b49a4d5da0b006dd1c: acquiring CA information for follower catalog manager: success
I20260929 10:54:27.148708 10956 catalog_manager.cc:1602] T 00000000000000000000000000000000 P be6a3c2e840a41b49a4d5da0b006dd1c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 10:54:27.236006 11006 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c0d388dae4d2405f9059bdc907070e30: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 10:54:27.317348 11072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:27.314580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49610 (local address 127.20.80.253:44085)
0929 10:54:27.314796 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:27.314799 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:27.314807 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:54:27.314844 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:27.314845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:27.314898 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:27.314988 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:54:27.315158 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:27.315677 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:27.316456 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:27.316666 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:27.316745 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:27.316935 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:54:27.317077 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:54:27.317168 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:27.317207 (+ 39us) server_negotiation.cc:299] Negotiation successful
0929 10:54:27.317232 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":149,"threads_started":1}
I20260929 10:54:27.322028 11078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:27.319216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60682 (local address 127.20.80.252:36567)
0929 10:54:27.319351 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:27.319353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:27.319363 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:54:27.319409 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:27.319409 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:27.319446 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:27.319519 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:54:27.319677 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:27.320216 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:27.321177 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:27.321384 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:27.321477 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:27.321658 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:54:27.321771 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:54:27.321877 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:27.321902 (+ 25us) server_negotiation.cc:299] Negotiation successful
0929 10:54:27.321928 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
I20260929 10:54:27.326586 11072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:27.323963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49624 (local address 127.20.80.253:44085)
0929 10:54:27.324003 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:27.324005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:27.324015 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:54:27.324045 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:27.324046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:27.324058 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:27.324115 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:54:27.324213 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:27.324761 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:27.325869 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:27.326087 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:27.326154 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:27.326294 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:54:27.326333 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:54:27.326397 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:27.326458 (+ 61us) server_negotiation.cc:299] Negotiation successful
0929 10:54:27.326482 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 10:54:27.327206 11085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:27.323697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42186 (local address 127.20.80.254:41381)
0929 10:54:27.323827 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:27.323829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:27.323998 (+ 169us) server_negotiation.cc:407] Connection header received
0929 10:54:27.324132 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:27.324133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:27.324183 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:27.324257 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:54:27.324449 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:27.325210 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:27.326115 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:27.326431 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:27.326492 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:27.326864 (+ 372us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:54:27.326950 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:54:27.326995 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:27.327079 (+ 84us) server_negotiation.cc:299] Negotiation successful
0929 10:54:27.327108 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":61,"threads_started":1}
I20260929 10:54:27.327401 11078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:27.323871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60686 (local address 127.20.80.252:36567)
0929 10:54:27.323916 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:27.323918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:27.324053 (+ 135us) server_negotiation.cc:407] Connection header received
0929 10:54:27.324076 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:27.324077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:27.324085 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:27.324129 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:54:27.324298 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:27.325022 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:27.326183 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:27.326474 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:27.326528 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:27.326828 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:54:27.327072 (+ 244us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:54:27.327122 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:27.327217 (+ 95us) server_negotiation.cc:299] Negotiation successful
0929 10:54:27.327239 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 10:54:27.328881 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 10720
I20260929 10:54:27.334880 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 10795
I20260929 10:54:27.340075 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 10871
2026-09-29T10:54:27Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2813 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:54:27 dist-test-slave-vqch krb5kdc[11089](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:54:27 dist-test-slave-vqch krb5kdc[11089](info): set up 2 sockets
Sep 29 10:54:27 dist-test-slave-vqch krb5kdc[11089](info): commencing operation
krb5kdc: starting...
W20260929 10:54:28.380893 20803 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/krb5kdc/test-user.keytab.
Sep 29 10:54:28 dist-test-slave-vqch krb5kdc[11089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:54:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:54:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.80.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
Entry for principal kudu/127.20.80.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab.
I20260929 10:54:28.499923 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:40567
--webserver_interface=127.20.80.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:42101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:40567,127.20.80.253:33893,127.20.80.252:37429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:54:28.582427 11102 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 10:54:28.582634 11102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:54:28.582652 11102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:54:28.582674 11102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:54:28.583781 11102 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:54:28.583815 11102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:54:28.583830 11102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:54:28.583842 11102 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:54:28.583854 11102 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:54:28.585448 11102 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:42101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:40567,127.20.80.253:33893,127.20.80.252:37429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.254.keytab
--principal=kudu/127.20.80.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.254:40567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.80.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:54:28.585638 11102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:54:28.585815 11102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 10:54:28.590631 11102 server_base.cc:1034] running on GCE node
W20260929 10:54:28.590598 11107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:28.590593 11108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:28.590584 11110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:28.591032 11102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:54:28.591331 11102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:54:28.592489 11102 hybrid_clock.cc:648] HybridClock initialized: now 1790679268592468 us; error 51 us; skew 500 ppm
Sep 29 10:54:28 dist-test-slave-vqch krb5kdc[11089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679268, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:54:28.599632 11102 init.cc:377] Logged in from keytab as kudu/127.20.80.254@KRBTEST.COM (short username kudu)
I20260929 10:54:28.600104 11102 webserver.cc:460] Webserver started at http://127.20.80.254:45735/ using document root <none> and password file <none>
I20260929 10:54:28.600317 11102 fs_manager.cc:359] Metadata directory not provided
I20260929 10:54:28.600358 11102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:54:28.600455 11102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:54:28.601570 11102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-0/data/instance:
uuid: "f6e94d1118144dd0b0b685473197984e"
format_stamp: "Formatted at 2026-09-29 10:54:28 on dist-test-slave-vqch"
server_key: "fb4122366616ccf8729c3ec908aa5a8c"
server_key_iv: "80cc6394a71312a72e080d31301f3f23"
server_key_version: "encryptionkey@0"
I20260929 10:54:28.601871 11102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-0/wal/instance:
uuid: "f6e94d1118144dd0b0b685473197984e"
format_stamp: "Formatted at 2026-09-29 10:54:28 on dist-test-slave-vqch"
server_key: "fb4122366616ccf8729c3ec908aa5a8c"
server_key_iv: "80cc6394a71312a72e080d31301f3f23"
server_key_version: "encryptionkey@0"
I20260929 10:54:28.603125 11102 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:54:28.603843 11117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:28.604034 11102 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:54:28.604125 11102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-0/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-0/wal
uuid: "f6e94d1118144dd0b0b685473197984e"
format_stamp: "Formatted at 2026-09-29 10:54:28 on dist-test-slave-vqch"
server_key: "fb4122366616ccf8729c3ec908aa5a8c"
server_key_iv: "80cc6394a71312a72e080d31301f3f23"
server_key_version: "encryptionkey@0"
I20260929 10:54:28.604189 11102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:54:28.615151 11102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:54:28.617179 11102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:54:28.617303 11102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:54:28.621258 11102 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.254:40567
I20260929 10:54:28.621347 11169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.254:40567 every 8 connection(s)
I20260929 10:54:28.621724 11102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-0/data/info.pb
I20260929 10:54:28.622745 11170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:54:28.624368 11170 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.80.254" port: 40567 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:54:28.624706 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 11102
I20260929 10:54:28.624831 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-0/wal/instance
I20260929 10:54:28.625102 20803 external_mini_cluster.cc:1428] Setting key d16b081c4c3ce6d258b614e3228070a6
Sep 29 10:54:28 dist-test-slave-vqch krb5kdc[11089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679268, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.80.253@KRBTEST.COM; defaulting to no policy
I20260929 10:54:28.635975 11173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.625145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:40567 (local address 127.0.0.1:47048)
0929 10:54:28.625354 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:28.625400 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 10:54:28.626136 (+ 736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:54:28.626472 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:54:28.626475 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:54:28.626844 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:54:28.627030 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:28.627035 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.628295 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.628297 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:28.629530 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:28.629533 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.630093 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.630094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:28.630191 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:28.630730 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:54:28.630740 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:54:28.632402 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:54:28.634339 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:28.634346 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:28.634348 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:28.634586 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:28.634895 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:28.634896 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:28.634897 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:28.634998 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:28.635380 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:54:28.635388 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:54:28.635474 (+ 86us) client_negotiation.cc:769] Sending connection context
0929 10:54:28.635566 (+ 92us) client_negotiation.cc:240] Negotiation successful
0929 10:54:28.635720 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":102,"threads_started":1}
I20260929 10:54:28.635977 11172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.625145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47048 (local address 127.20.80.254:40567)
0929 10:54:28.625360 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:28.625370 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:28.625484 (+ 114us) server_negotiation.cc:407] Connection header received
0929 10:54:28.626329 (+ 845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:28.626331 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:28.626348 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:28.626431 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:28.627135 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.628187 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.629667 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.629979 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.630066 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:28.632486 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:28.632495 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:28.632495 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:28.632503 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:28.634246 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:28.634718 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:28.634721 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:28.634721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:28.634795 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:28.635068 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:28.635069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:28.635070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:28.635271 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:28.635357 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:28.635637 (+ 280us) server_negotiation.cc:299] Negotiation successful
0929 10:54:28.635777 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":170,"thread_start_us":105,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.253@KRBTEST.COM" created.
I20260929 10:54:28.637141 11170 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.80.253" port: 33893 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:54:28.637511 11173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.637385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:33893 (local address 127.0.0.1:53892)
0929 10:54:28.637436 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:28.637482 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.253:33893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260929 10:54:28.637671 11119 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.80.253:33893: connect: Connection refused (error 111)
I20260929 10:54:28.638127 11173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.638026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:33893 (local address 127.0.0.1:53906)
0929 10:54:28.638076 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:28.638100 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.253:33893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260929 10:54:28.638253 11170 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.80.253:33893: Network error: Client connection negotiation failed: client connection to 127.20.80.253:33893: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.253.keytab.
Entry for principal kudu/127.20.80.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.253.keytab.
I20260929 10:54:28.651675 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.253:33893
--webserver_interface=127.20.80.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.253.keytab
--principal=kudu/127.20.80.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:42101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:40567,127.20.80.253:33893,127.20.80.252:37429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:54:28.687502 11170 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.80.253" port: 33893 } attempt: 1
I20260929 10:54:28.688179 11173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.687966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:33893 (local address 127.0.0.1:53918)
0929 10:54:28.688064 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:28.688127 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.253:33893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260929 10:54:28.688687 11173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.688527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:33893 (local address 127.0.0.1:53920)
0929 10:54:28.688629 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:28.688662 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.253:33893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260929 10:54:28.688828 11170 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.80.253:33893: Network error: Client connection negotiation failed: client connection to 127.20.80.253:33893: connect: Connection refused (error 111)
W20260929 10:54:28.733340 11177 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 10:54:28.733546 11177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:54:28.733562 11177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:54:28.733582 11177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:54:28.734638 11177 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:54:28.734670 11177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:54:28.734684 11177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:54:28.734696 11177 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:54:28.734709 11177 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:54:28.736150 11177 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:42101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:40567,127.20.80.253:33893,127.20.80.252:37429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.253.keytab
--principal=kudu/127.20.80.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.253:33893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.80.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:54:28.736332 11177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:54:28.736492 11177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:54:28.741401 11182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:28.741503 11183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:28.741582 11177 server_base.cc:1034] running on GCE node
W20260929 10:54:28.741417 11185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:28.741974 11177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:54:28.742233 11177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:54:28.743376 11177 hybrid_clock.cc:648] HybridClock initialized: now 1790679268743355 us; error 39 us; skew 500 ppm
Sep 29 10:54:28 dist-test-slave-vqch krb5kdc[11089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679268, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:54:28.745851 11177 init.cc:377] Logged in from keytab as kudu/127.20.80.253@KRBTEST.COM (short username kudu)
I20260929 10:54:28.746316 11177 webserver.cc:460] Webserver started at http://127.20.80.253:43701/ using document root <none> and password file <none>
I20260929 10:54:28.746500 11177 fs_manager.cc:359] Metadata directory not provided
I20260929 10:54:28.746538 11177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:54:28.746662 11177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:54:28.747752 11177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-1/data/instance:
uuid: "512de9c5d1ac47febcbdfed527afb630"
format_stamp: "Formatted at 2026-09-29 10:54:28 on dist-test-slave-vqch"
server_key: "daabf9b95d67e0d597b61d3be0767b55"
server_key_iv: "fc32d648c67ff7d8495dfeeed6831c83"
server_key_version: "encryptionkey@0"
I20260929 10:54:28.748049 11177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-1/wal/instance:
uuid: "512de9c5d1ac47febcbdfed527afb630"
format_stamp: "Formatted at 2026-09-29 10:54:28 on dist-test-slave-vqch"
server_key: "daabf9b95d67e0d597b61d3be0767b55"
server_key_iv: "fc32d648c67ff7d8495dfeeed6831c83"
server_key_version: "encryptionkey@0"
I20260929 10:54:28.749341 11177 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 10:54:28.750068 11192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:28.750257 11177 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:54:28.750342 11177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-1/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-1/wal
uuid: "512de9c5d1ac47febcbdfed527afb630"
format_stamp: "Formatted at 2026-09-29 10:54:28 on dist-test-slave-vqch"
server_key: "daabf9b95d67e0d597b61d3be0767b55"
server_key_iv: "fc32d648c67ff7d8495dfeeed6831c83"
server_key_version: "encryptionkey@0"
I20260929 10:54:28.750417 11177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:54:28.757011 11170 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.80.253" port: 33893 } attempt: 2
I20260929 10:54:28.757454 11173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.757313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:33893 (local address 127.0.0.1:53922)
0929 10:54:28.757373 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:28.757416 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.253:33893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260929 10:54:28.757782 11173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.757698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:33893 (local address 127.0.0.1:53936)
0929 10:54:28.757739 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:28.757765 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.253:33893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260929 10:54:28.757903 11170 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.80.253:33893: Network error: Client connection negotiation failed: client connection to 127.20.80.253:33893: connect: Connection refused (error 111)
I20260929 10:54:28.771420 11177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:54:28.773844 11177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:54:28.774001 11177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:54:28.777983 11177 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.253:33893
I20260929 10:54:28.778024 11244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.253:33893 every 8 connection(s)
I20260929 10:54:28.778386 11177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-1/data/info.pb
I20260929 10:54:28.779404 11245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:54:28.780916 11245 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.80.254" port: 40567 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 10:54:28 dist-test-slave-vqch krb5kdc[11089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679268, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.253@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:54:28.786664 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 11177
I20260929 10:54:28.786779 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-1/wal/instance
I20260929 10:54:28.787012 20803 external_mini_cluster.cc:1428] Setting key f081d393774dcaffbd9c3711ca5c517f
I20260929 10:54:28.790320 11172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.781744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47052 (local address 127.20.80.254:40567)
0929 10:54:28.781831 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:28.781834 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:28.781980 (+ 146us) server_negotiation.cc:407] Connection header received
0929 10:54:28.782527 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:28.782528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:28.782543 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:28.782600 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:28.783261 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.783727 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.784667 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.784858 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.784907 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:28.787163 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:28.787165 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:28.787165 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:28.787179 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:28.788851 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:28.789303 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:28.789304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:28.789305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:28.789378 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:28.789613 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:28.789614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:28.789615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:28.789766 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:28.789835 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:28.790073 (+ 238us) server_negotiation.cc:299] Negotiation successful
0929 10:54:28.790169 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 10:54:28.790331 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.781650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:40567 (local address 127.0.0.1:47052)
0929 10:54:28.781847 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:28.781887 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 10:54:28.782414 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:54:28.782711 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:54:28.782715 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:54:28.782972 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:54:28.783175 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:28.783180 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.783806 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.783807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:28.784595 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:28.784597 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.784936 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.784937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:28.785105 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:28.785546 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:54:28.785554 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:54:28.787068 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:54:28.788941 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:28.788948 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:28.788950 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:28.789217 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:28.789449 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:28.789450 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:28.789450 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:28.789549 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:28.789831 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:54:28.789835 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:54:28.789948 (+ 113us) client_negotiation.cc:769] Sending connection context
0929 10:54:28.790031 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 10:54:28.790152 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":78,"threads_started":1}
I20260929 10:54:28.790966 11245 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.80.253" port: 33893 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.20.80.252@KRBTEST.COM; defaulting to no policy
Sep 29 10:54:28 dist-test-slave-vqch krb5kdc[11089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679268, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.253@KRBTEST.COM for kudu/127.20.80.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.80.252@KRBTEST.COM" created.
I20260929 10:54:28.798588 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.791246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:33893 (local address 127.0.0.1:53944)
0929 10:54:28.791304 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:28.791315 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:54:28.791353 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:54:28.791621 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:54:28.791621 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:54:28.791765 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:54:28.791809 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:28.791810 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.792681 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.792682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:28.793250 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:28.793252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.793728 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.793729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:28.793794 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:28.794270 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:54:28.794274 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:54:28.795410 (+ 1136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:54:28.797446 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:28.797447 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:28.797448 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:28.797676 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:28.797914 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:28.797915 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:28.797915 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:28.797968 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:28.798332 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:54:28.798333 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:54:28.798374 (+ 41us) client_negotiation.cc:769] Sending connection context
0929 10:54:28.798443 (+ 69us) client_negotiation.cc:240] Negotiation successful
0929 10:54:28.798520 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260929 10:54:28.798794 11249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.791333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53944 (local address 127.20.80.253:33893)
0929 10:54:28.791492 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:28.791496 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:28.791512 (+ 16us) server_negotiation.cc:407] Connection header received
0929 10:54:28.791529 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:28.791530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:28.791541 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:28.791620 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:28.791883 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.792590 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.793344 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.793643 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.793730 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:28.795502 (+ 1772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:28.795512 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:28.795512 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:28.795524 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:28.797344 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:28.797774 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:28.797777 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:28.797778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:28.797846 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:28.798030 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:28.798031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:28.798032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:28.798243 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:28.798333 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:28.798499 (+ 166us) server_negotiation.cc:299] Negotiation successful
0929 10:54:28.798589 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":84,"threads_started":1}
I20260929 10:54:28.799472 11245 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37429 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:54:28.799881 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.799757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:37429 (local address 127.0.0.1:41298)
0929 10:54:28.799804 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:28.799854 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.252:37429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260929 10:54:28.800040 11193 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.80.252:37429: connect: Connection refused (error 111)
I20260929 10:54:28.800536 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.800414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:37429 (local address 127.0.0.1:41300)
0929 10:54:28.800456 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:28.800488 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.252:37429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260929 10:54:28.800767 11245 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.80.252:37429: Network error: Client connection negotiation failed: client connection to 127.20.80.252:37429: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.80.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.252.keytab.
Entry for principal kudu/127.20.80.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.252.keytab.
I20260929 10:54:28.812475 20803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
/tmp/dist-test-taskfQc4Oq/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.252:37429
--webserver_interface=127.20.80.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.252.keytab
--principal=kudu/127.20.80.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.80.212:42101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.80.254:40567,127.20.80.253:33893,127.20.80.252:37429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:54:28.849105 11170 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.80.253" port: 33893 } attempt: 3
I20260929 10:54:28.850013 11245 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37429 } attempt: 1
I20260929 10:54:28.850384 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.850258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:37429 (local address 127.0.0.1:41304)
0929 10:54:28.850322 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:28.850356 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.252:37429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260929 10:54:28.850796 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.850700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:37429 (local address 127.0.0.1:41320)
0929 10:54:28.850746 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:28.850772 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.252:37429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260929 10:54:28.850920 11245 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.80.252:37429: Network error: Client connection negotiation failed: client connection to 127.20.80.252:37429: connect: Connection refused (error 111)
Sep 29 10:54:28 dist-test-slave-vqch krb5kdc[11089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679268, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for kudu/127.20.80.253@KRBTEST.COM
I20260929 10:54:28.856755 11173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.849496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:33893 (local address 127.0.0.1:53950)
0929 10:54:28.849571 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:28.849595 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 10:54:28.849680 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:54:28.849822 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:54:28.849823 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:54:28.850009 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:54:28.850094 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:28.850096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.851059 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.851060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:28.851596 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:28.851597 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.852085 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.852086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:28.852147 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:28.852631 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:54:28.852639 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:54:28.853941 (+ 1302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:54:28.855767 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:28.855768 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:28.855769 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:28.855940 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:28.856144 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:28.856145 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:28.856146 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:28.856209 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:28.856474 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:54:28.856475 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:54:28.856514 (+ 39us) client_negotiation.cc:769] Sending connection context
0929 10:54:28.856573 (+ 59us) client_negotiation.cc:240] Negotiation successful
0929 10:54:28.856686 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260929 10:54:28.856778 11249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.849591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53950 (local address 127.20.80.253:33893)
0929 10:54:28.849667 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:28.849669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:28.849682 (+ 13us) server_negotiation.cc:407] Connection header received
0929 10:54:28.849735 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:28.849736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:28.849747 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:28.849807 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:28.850211 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.850987 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.851675 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.852006 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.852088 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:28.854065 (+ 1977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:28.854066 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:28.854067 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:28.854074 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:28.855680 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:28.856014 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:28.856015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:28.856015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:28.856076 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:28.856279 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:28.856280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:28.856280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:28.856412 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:28.856478 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:28.856616 (+ 138us) server_negotiation.cc:299] Negotiation successful
0929 10:54:28.856693 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 10:54:28.857278 11170 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37429 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:54:28.857609 11173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.857500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:37429 (local address 127.0.0.1:41328)
0929 10:54:28.857554 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:28.857585 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.252:37429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260929 10:54:28.857893 11173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.857809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:37429 (local address 127.0.0.1:41336)
0929 10:54:28.857852 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:28.857876 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.252:37429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260929 10:54:28.857964 11170 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.80.252:37429: Network error: Client connection negotiation failed: client connection to 127.20.80.252:37429: connect: Connection refused (error 111)
I20260929 10:54:28.889101 11170 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37429 } attempt: 1
I20260929 10:54:28.889626 11173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.889474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:37429 (local address 127.0.0.1:41350)
0929 10:54:28.889544 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:28.889589 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.252:37429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260929 10:54:28.890029 11173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.889922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:37429 (local address 127.0.0.1:41356)
0929 10:54:28.889973 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:28.890003 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.252:37429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260929 10:54:28.890148 11170 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.80.252:37429: Network error: Client connection negotiation failed: client connection to 127.20.80.252:37429: connect: Connection refused (error 111)
W20260929 10:54:28.895071 11252 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 10:54:28.895262 11252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:54:28.895278 11252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:54:28.895337 11252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:54:28.896453 11252 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:54:28.896493 11252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:54:28.896508 11252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:54:28.896520 11252 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:54:28.896533 11252 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:54:28.898059 11252 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.80.212:42101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.80.254:40567,127.20.80.253:33893,127.20.80.252:37429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/krb5kdc/kudu_127.20.80.252.keytab
--principal=kudu/127.20.80.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.80.252:37429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.80.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:54:28.898300 11252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:54:28.898530 11252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:54:28.903414 11260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:28.903409 11258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:54:28.903519 11257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:54:28.903618 11252 server_base.cc:1034] running on GCE node
I20260929 10:54:28.903848 11252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:54:28.904160 11252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:54:28.905306 11252 hybrid_clock.cc:648] HybridClock initialized: now 1790679268905321 us; error 57 us; skew 500 ppm
Sep 29 10:54:28 dist-test-slave-vqch krb5kdc[11089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679268, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:54:28.908057 11252 init.cc:377] Logged in from keytab as kudu/127.20.80.252@KRBTEST.COM (short username kudu)
I20260929 10:54:28.908493 11252 webserver.cc:460] Webserver started at http://127.20.80.252:38031/ using document root <none> and password file <none>
I20260929 10:54:28.908727 11252 fs_manager.cc:359] Metadata directory not provided
I20260929 10:54:28.908773 11252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:54:28.909029 11252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:54:28.910164 11252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-2/data/instance:
uuid: "dcaf5a98737c4086927dcc5fcc7f5d31"
format_stamp: "Formatted at 2026-09-29 10:54:28 on dist-test-slave-vqch"
server_key: "5e5d03cd5ddc7618d5abfe983ee1c0a9"
server_key_iv: "ce00fd317f0c69507eeb53c4a55a2229"
server_key_version: "encryptionkey@0"
I20260929 10:54:28.910550 11252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-2/wal/instance:
uuid: "dcaf5a98737c4086927dcc5fcc7f5d31"
format_stamp: "Formatted at 2026-09-29 10:54:28 on dist-test-slave-vqch"
server_key: "5e5d03cd5ddc7618d5abfe983ee1c0a9"
server_key_iv: "ce00fd317f0c69507eeb53c4a55a2229"
server_key_version: "encryptionkey@0"
I20260929 10:54:28.912503 11252 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260929 10:54:28.913554 11267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:54:28.913766 11252 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 10:54:28.913884 11252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-2/data,/tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-2/wal
uuid: "dcaf5a98737c4086927dcc5fcc7f5d31"
format_stamp: "Formatted at 2026-09-29 10:54:28 on dist-test-slave-vqch"
server_key: "5e5d03cd5ddc7618d5abfe983ee1c0a9"
server_key_iv: "ce00fd317f0c69507eeb53c4a55a2229"
server_key_version: "encryptionkey@0"
I20260929 10:54:28.913965 11252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:54:28.919052 11245 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37429 } attempt: 2
I20260929 10:54:28.919492 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.919358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:37429 (local address 127.0.0.1:41364)
0929 10:54:28.919420 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:28.919462 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.252:37429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260929 10:54:28.919935 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.919826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:37429 (local address 127.0.0.1:41368)
0929 10:54:28.919873 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:28.919901 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.80.252:37429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260929 10:54:28.920039 11245 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.80.252:37429: Network error: Client connection negotiation failed: client connection to 127.20.80.252:37429: connect: Connection refused (error 111)
I20260929 10:54:28.940106 11252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:54:28.942793 11252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:54:28.942960 11252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:54:28.948299 11252 rpc_server.cc:307] RPC server started. Bound to: 127.20.80.252:37429
I20260929 10:54:28.948336 11321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.80.252:37429 every 8 connection(s)
I20260929 10:54:28.948788 11252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-2/data/info.pb
I20260929 10:54:28.949855 11322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:54:28.951448 11322 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.80.254" port: 40567 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:54:28.957588 20803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu as pid 11252
I20260929 10:54:28.957702 20803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfQc4Oq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790678996126930-20803-0/minicluster-data/master-2/wal/instance
I20260929 10:54:28.957959 20803 external_mini_cluster.cc:1428] Setting key 747729e777f65c32ff81d4b214cbea83
Sep 29 10:54:28 dist-test-slave-vqch krb5kdc[11089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679268, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.252@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:54:28.962802 11172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.952221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47060 (local address 127.20.80.254:40567)
0929 10:54:28.952272 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:28.952274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:28.952527 (+ 253us) server_negotiation.cc:407] Connection header received
0929 10:54:28.953007 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:28.953009 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:28.953024 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:28.953098 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:28.953750 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.954655 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.955715 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.956092 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.956171 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:28.959009 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:28.959011 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:28.959011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:28.959022 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:28.960893 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:28.961660 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:28.961661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:28.961661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:28.961726 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:28.961999 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:28.962000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:28.962001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:28.962151 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:28.962213 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:28.962609 (+ 396us) server_negotiation.cc:299] Negotiation successful
0929 10:54:28.962707 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 10:54:28.963398 11324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.952201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.254:40567 (local address 127.0.0.1:47060)
0929 10:54:28.952408 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:28.952445 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 10:54:28.952891 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:54:28.953203 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:54:28.953221 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:54:28.953467 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:54:28.953661 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:28.953666 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.954766 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.954767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:28.955618 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:28.955621 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.956218 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.956219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:28.956349 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:28.956961 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:54:28.956969 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:54:28.958841 (+ 1872us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:54:28.961254 (+ 2413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:28.961259 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:28.961261 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:28.961567 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:28.961794 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:28.961795 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:28.961795 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:28.961920 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:28.962269 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:54:28.962275 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:54:28.962440 (+ 165us) client_negotiation.cc:769] Sending connection context
0929 10:54:28.962940 (+ 500us) client_negotiation.cc:240] Negotiation successful
0929 10:54:28.963128 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":122,"threads_started":1}
Sep 29 10:54:28 dist-test-slave-vqch krb5kdc[11089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.252@KRBTEST.COM
I20260929 10:54:28.965332 11170 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37429 } attempt: 2
I20260929 10:54:28.968966 11326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.958526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41374 (local address 127.20.80.252:37429)
0929 10:54:28.958799 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:28.958805 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:28.958827 (+ 22us) server_negotiation.cc:407] Connection header received
0929 10:54:28.958852 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:28.958854 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:28.958904 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:28.958983 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:28.959907 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.961127 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.962183 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.962626 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.962688 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:28.965541 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:28.965551 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:28.965551 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:28.965559 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:28.967378 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:28.967784 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:28.967787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:28.967787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:28.967844 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:28.968022 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:28.968023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:28.968023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:28.968190 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:28.968324 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:28.968785 (+ 461us) server_negotiation.cc:299] Negotiation successful
0929 10:54:28.968861 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":80,"threads_started":1}
I20260929 10:54:28.969889 11322 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.80.253" port: 33893 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 10:54:28 dist-test-slave-vqch krb5kdc[11089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679268, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.254@KRBTEST.COM for kudu/127.20.80.252@KRBTEST.COM
Sep 29 10:54:28 dist-test-slave-vqch krb5kdc[11089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679268, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.252@KRBTEST.COM for kudu/127.20.80.253@KRBTEST.COM
Sep 29 10:54:28 dist-test-slave-vqch krb5kdc[11089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.254@KRBTEST.COM
I20260929 10:54:28.978111 11249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.970222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53952 (local address 127.20.80.253:33893)
0929 10:54:28.970263 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:28.970265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:28.970331 (+ 66us) server_negotiation.cc:407] Connection header received
0929 10:54:28.970421 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:28.970422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:28.970432 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:28.970459 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:28.970849 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.971623 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.972732 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.973121 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.973162 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:28.975231 (+ 2069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:28.975232 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:28.975232 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:28.975240 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:28.976926 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:28.977350 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:28.977352 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:28.977352 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:28.977408 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:28.977587 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:28.977587 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:28.977588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:28.977715 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:28.977753 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:28.977941 (+ 188us) server_negotiation.cc:299] Negotiation successful
0929 10:54:28.978013 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 10:54:28.978612 11324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.970130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.253:33893 (local address 127.0.0.1:53952)
0929 10:54:28.970289 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:28.970298 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:54:28.970352 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:54:28.970517 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:54:28.970518 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:54:28.970684 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:54:28.970749 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:28.970751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.971724 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.971725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:28.972622 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:28.972624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.973236 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.973237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:28.973311 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:28.973650 (+ 339us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:54:28.973656 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:54:28.975145 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:54:28.977040 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:28.977041 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:28.977041 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:28.977262 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:28.977475 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:28.977476 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:28.977476 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:28.977526 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:28.977800 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:54:28.977801 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:54:28.977841 (+ 40us) client_negotiation.cc:769] Sending connection context
0929 10:54:28.978232 (+ 391us) client_negotiation.cc:240] Negotiation successful
0929 10:54:28.978525 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145}
I20260929 10:54:28.979054 11322 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37429 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:54:28.980203 11172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.969999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47072 (local address 127.20.80.254:40567)
0929 10:54:28.970034 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:28.970036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:28.970168 (+ 132us) server_negotiation.cc:407] Connection header received
0929 10:54:28.970258 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:28.970259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:28.970268 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:28.970294 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:28.970892 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.971604 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.972583 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.972874 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.972918 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:28.977101 (+ 4183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:28.977102 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:28.977103 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:28.977110 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:28.978841 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:28.979258 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:28.979260 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:28.979260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:28.979318 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:28.979510 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:28.979512 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:28.979512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:28.979668 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:28.979841 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:28.980031 (+ 190us) server_negotiation.cc:299] Negotiation successful
0929 10:54:28.980108 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":18}
I20260929 10:54:28.980377 11173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.969209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:37429 (local address 127.0.0.1:41388)
0929 10:54:28.969432 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:28.969443 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:54:28.969489 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:54:28.969664 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:54:28.969666 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:54:28.969838 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:54:28.969885 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:28.969887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.970844 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.970845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:28.971660 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:28.971662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.972124 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.972125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:28.972196 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:28.972633 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:54:28.972638 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:54:28.975840 (+ 3202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:54:28.979326 (+ 3486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:28.979327 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:28.979327 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:28.979541 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:28.979770 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:28.979771 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:28.979772 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:28.979830 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:28.980105 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:54:28.980106 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:54:28.980149 (+ 43us) client_negotiation.cc:769] Sending connection context
0929 10:54:28.980195 (+ 46us) client_negotiation.cc:240] Negotiation successful
0929 10:54:28.980286 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200}
I20260929 10:54:28.981206 11327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.969358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41388 (local address 127.20.80.252:37429)
0929 10:54:28.969533 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:28.969535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:28.969543 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:54:28.969566 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:28.969567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:28.969578 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:28.969655 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:28.969970 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.970747 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.971748 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.972045 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.973229 (+ 1184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:28.975941 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:28.975942 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:28.975942 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:28.975949 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:28.979241 (+ 3292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:28.979624 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:28.979624 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:28.979625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:28.979695 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:28.979903 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:28.979904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:28.979904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:28.980032 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:28.980865 (+ 833us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:28.980906 (+ 41us) server_negotiation.cc:299] Negotiation successful
0929 10:54:28.981034 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":117,"threads_started":1}
I20260929 10:54:28.983343 11170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6e94d1118144dd0b0b685473197984e: Bootstrap starting.
Sep 29 10:54:28 dist-test-slave-vqch krb5kdc[11089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679268, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.252@KRBTEST.COM for kudu/127.20.80.252@KRBTEST.COM
I20260929 10:54:28.984894 11170 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6e94d1118144dd0b0b685473197984e: Neither blocks nor log segments found. Creating new log.
I20260929 10:54:28.985280 11170 log.cc:824] T 00000000000000000000000000000000 P f6e94d1118144dd0b0b685473197984e: Log is configured to *not* fsync() on all Append() calls
I20260929 10:54:28.987651 11170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6e94d1118144dd0b0b685473197984e: No bootstrap required, opened a new log
I20260929 10:54:28.988276 11324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.979314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:37429 (local address 127.0.0.1:41396)
0929 10:54:28.979380 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:28.979390 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:54:28.979420 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:54:28.979746 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:54:28.979746 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:54:28.979894 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:54:28.979930 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:28.979932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.980811 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.980812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:28.981626 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:28.981629 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.982087 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.982088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:28.982143 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:28.982558 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:54:28.982563 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:54:28.984048 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:54:28.987007 (+ 2959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:28.987008 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:28.987008 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:28.987239 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:28.987436 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:28.987436 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:28.987437 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:28.987486 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:28.987879 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:54:28.987880 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:54:28.987932 (+ 52us) client_negotiation.cc:769] Sending connection context
0929 10:54:28.988012 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 10:54:28.988121 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260929 10:54:28.988323 11326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.979561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41396 (local address 127.20.80.252:37429)
0929 10:54:28.979602 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:28.979604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:28.979611 (+ 7us) server_negotiation.cc:407] Connection header received
0929 10:54:28.979645 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:28.979645 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:28.979656 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:28.979707 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:28.979999 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.980728 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.981690 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.982006 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.982081 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:28.984130 (+ 2049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:28.984131 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:28.984132 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:28.984140 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:28.986304 (+ 2164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:28.987321 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:28.987322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:28.987323 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:28.987372 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:28.987542 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:28.987543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:28.987543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:28.987801 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:28.987878 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:28.988052 (+ 174us) server_negotiation.cc:299] Negotiation successful
0929 10:54:28.988181 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":22}
I20260929 10:54:28.990072 11322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcaf5a98737c4086927dcc5fcc7f5d31: Bootstrap starting.
I20260929 10:54:28.990092 11170 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6e94d1118144dd0b0b685473197984e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e94d1118144dd0b0b685473197984e" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 40567 } } peers { permanent_uuid: "512de9c5d1ac47febcbdfed527afb630" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 33893 } } peers { permanent_uuid: "dcaf5a98737c4086927dcc5fcc7f5d31" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37429 } }
I20260929 10:54:28.990242 11170 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6e94d1118144dd0b0b685473197984e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e94d1118144dd0b0b685473197984e, State: Initialized, Role: FOLLOWER
I20260929 10:54:28.990325 11170 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6e94d1118144dd0b0b685473197984e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e94d1118144dd0b0b685473197984e" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 40567 } } peers { permanent_uuid: "512de9c5d1ac47febcbdfed527afb630" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 33893 } } peers { permanent_uuid: "dcaf5a98737c4086927dcc5fcc7f5d31" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37429 } }
I20260929 10:54:28.990658 11170 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6e94d1118144dd0b0b685473197984e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:54:28.990617 11329 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6e94d1118144dd0b0b685473197984e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e94d1118144dd0b0b685473197984e" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 40567 } } peers { permanent_uuid: "512de9c5d1ac47febcbdfed527afb630" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 33893 } } peers { permanent_uuid: "dcaf5a98737c4086927dcc5fcc7f5d31" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37429 } } }
I20260929 10:54:28.990818 11322 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dcaf5a98737c4086927dcc5fcc7f5d31: Neither blocks nor log segments found. Creating new log.
I20260929 10:54:28.990844 11329 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6e94d1118144dd0b0b685473197984e [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:54:28.991163 11322 log.cc:824] T 00000000000000000000000000000000 P dcaf5a98737c4086927dcc5fcc7f5d31: Log is configured to *not* fsync() on all Append() calls
I20260929 10:54:28.992048 11322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcaf5a98737c4086927dcc5fcc7f5d31: No bootstrap required, opened a new log
W20260929 10:54:28.992684 11341 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f6e94d1118144dd0b0b685473197984e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:54:28.992797 11341 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:54:28.994201 11322 raft_consensus.cc:348] T 00000000000000000000000000000000 P dcaf5a98737c4086927dcc5fcc7f5d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e94d1118144dd0b0b685473197984e" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 40567 } } peers { permanent_uuid: "512de9c5d1ac47febcbdfed527afb630" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 33893 } } peers { permanent_uuid: "dcaf5a98737c4086927dcc5fcc7f5d31" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37429 } }
I20260929 10:54:28.994360 11322 raft_consensus.cc:729] T 00000000000000000000000000000000 P dcaf5a98737c4086927dcc5fcc7f5d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcaf5a98737c4086927dcc5fcc7f5d31, State: Initialized, Role: FOLLOWER
I20260929 10:54:28.994449 11322 consensus_queue.cc:260] T 00000000000000000000000000000000 P dcaf5a98737c4086927dcc5fcc7f5d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e94d1118144dd0b0b685473197984e" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 40567 } } peers { permanent_uuid: "512de9c5d1ac47febcbdfed527afb630" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 33893 } } peers { permanent_uuid: "dcaf5a98737c4086927dcc5fcc7f5d31" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37429 } }
I20260929 10:54:28.994727 11343 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcaf5a98737c4086927dcc5fcc7f5d31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e94d1118144dd0b0b685473197984e" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 40567 } } peers { permanent_uuid: "512de9c5d1ac47febcbdfed527afb630" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 33893 } } peers { permanent_uuid: "dcaf5a98737c4086927dcc5fcc7f5d31" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37429 } } }
I20260929 10:54:28.994792 11322 sys_catalog.cc:564] T 00000000000000000000000000000000 P dcaf5a98737c4086927dcc5fcc7f5d31 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:54:28.994820 11343 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcaf5a98737c4086927dcc5fcc7f5d31 [sys.catalog]: This master's current role is: FOLLOWER
Sep 29 10:54:28 dist-test-slave-vqch krb5kdc[11089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.80.253@KRBTEST.COM
I20260929 10:54:28.995008 11102 master_runner.cc:186] Error getting master registration for 127.20.80.253:33893: OK, instance_id { permanent_uuid: "512de9c5d1ac47febcbdfed527afb630" instance_seqno: 1790679268776854 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 10:54:28.998027 11102 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260929 10:54:28.999148 11354 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dcaf5a98737c4086927dcc5fcc7f5d31: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:54:28.999218 11354 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:54:28.999797 11249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:28.981260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53960 (local address 127.20.80.253:33893)
0929 10:54:28.981316 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:28.981318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:28.981403 (+ 85us) server_negotiation.cc:407] Connection header received
0929 10:54:28.981491 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:28.981492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:28.981501 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:28.981530 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:28.986123 (+ 4593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.986905 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.993108 (+ 6203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:28.993444 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:28.993492 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:28.996552 (+ 3060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:28.996553 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:28.996554 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:28.996561 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:28.998188 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:28.998567 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:28.998569 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:28.998569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:28.998635 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:28.998857 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:28.998858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:28.998858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:28.999005 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:28.999159 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:28.999579 (+ 420us) server_negotiation.cc:299] Negotiation successful
0929 10:54:28.999676 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 10:54:29.001230 20803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:54:29.002154 11144 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "f6e94d1118144dd0b0b685473197984e"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47072
I20260929 10:54:29.002282 11144 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6e94d1118144dd0b0b685473197984e [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260929 10:54:29.002327 11144 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6e94d1118144dd0b0b685473197984e [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:54:29.000434 11252 master_runner.cc:186] Error getting master registration for 127.20.80.253:33893: OK, instance_id { permanent_uuid: "512de9c5d1ac47febcbdfed527afb630" instance_seqno: 1790679268776854 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 10:54:29.002789 11252 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 10:54:29.003175 11144 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6e94d1118144dd0b0b685473197984e [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e94d1118144dd0b0b685473197984e" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 40567 } } peers { permanent_uuid: "512de9c5d1ac47febcbdfed527afb630" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 33893 } } peers { permanent_uuid: "dcaf5a98737c4086927dcc5fcc7f5d31" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37429 } }
I20260929 10:54:29.003379 11144 leader_election.cc:288] T 00000000000000000000000000000000 P f6e94d1118144dd0b0b685473197984e [CANDIDATE]: Term 1 election: Requested vote from peers 512de9c5d1ac47febcbdfed527afb630 (127.20.80.253:33893), dcaf5a98737c4086927dcc5fcc7f5d31 (127.20.80.252:37429)
I20260929 10:54:29.009191 11295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f6e94d1118144dd0b0b685473197984e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "dcaf5a98737c4086927dcc5fcc7f5d31"
I20260929 10:54:29.009330 11295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dcaf5a98737c4086927dcc5fcc7f5d31 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:54:29.010366 11295 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dcaf5a98737c4086927dcc5fcc7f5d31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e94d1118144dd0b0b685473197984e in term 1.
I20260929 10:54:29.010623 11118 leader_election.cc:302] T 00000000000000000000000000000000 P f6e94d1118144dd0b0b685473197984e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dcaf5a98737c4086927dcc5fcc7f5d31, f6e94d1118144dd0b0b685473197984e; no voters:
I20260929 10:54:29.010769 11329 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6e94d1118144dd0b0b685473197984e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:54:29.010952 11329 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6e94d1118144dd0b0b685473197984e [term 1 LEADER]: Becoming Leader. State: Replica: f6e94d1118144dd0b0b685473197984e, State: Running, Role: LEADER
I20260929 10:54:29.011056 11329 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6e94d1118144dd0b0b685473197984e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e94d1118144dd0b0b685473197984e" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 40567 } } peers { permanent_uuid: "512de9c5d1ac47febcbdfed527afb630" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 33893 } } peers { permanent_uuid: "dcaf5a98737c4086927dcc5fcc7f5d31" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37429 } }
I20260929 10:54:29.011179 11245 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37429 } attempt: 3
I20260929 10:54:29.013309 11361 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6e94d1118144dd0b0b685473197984e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6e94d1118144dd0b0b685473197984e. Latest consensus state: current_term: 1 leader_uuid: "f6e94d1118144dd0b0b685473197984e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e94d1118144dd0b0b685473197984e" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 40567 } } peers { permanent_uuid: "512de9c5d1ac47febcbdfed527afb630" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 33893 } } peers { permanent_uuid: "dcaf5a98737c4086927dcc5fcc7f5d31" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37429 } } }
I20260929 10:54:29.013422 11361 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6e94d1118144dd0b0b685473197984e [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:29.015055 11363 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 29 10:54:29 dist-test-slave-vqch krb5kdc[11089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679268, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.80.253@KRBTEST.COM for kudu/127.20.80.252@KRBTEST.COM
I20260929 10:54:29.019972 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:29.011450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.80.252:37429 (local address 127.0.0.1:41408)
0929 10:54:29.011549 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0929 10:54:29.011560 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:54:29.011617 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:54:29.011824 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:54:29.011825 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:54:29.012033 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:54:29.012088 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:29.012091 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:29.013078 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:29.013080 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:29.013903 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:54:29.013906 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:54:29.014675 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:54:29.014676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:54:29.014753 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:29.015258 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:54:29.015263 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:54:29.016866 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:54:29.018780 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:29.018781 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:29.018781 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:29.019012 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:29.019264 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:54:29.019265 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:54:29.019266 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:54:29.019336 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:54:29.019648 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:54:29.019649 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:54:29.019701 (+ 52us) client_negotiation.cc:769] Sending connection context
0929 10:54:29.019763 (+ 62us) client_negotiation.cc:240] Negotiation successful
0929 10:54:29.019866 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260929 10:54:29.020602 11327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:29.011508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41408 (local address 127.20.80.252:37429)
0929 10:54:29.011692 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:29.011694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:29.011701 (+ 7us) server_negotiation.cc:407] Connection header received
0929 10:54:29.011724 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:29.011725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:29.011735 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:29.011779 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:29.012182 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:29.012921 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:29.014166 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:29.014534 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:29.014601 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:29.016960 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:29.016962 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:29.016962 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:29.016969 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:29.018681 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:29.019112 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:29.019113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:29.019113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:29.019179 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:29.019408 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:29.019409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:29.019409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:29.019549 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:29.019607 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:29.020422 (+ 815us) server_negotiation.cc:299] Negotiation successful
0929 10:54:29.020512 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164}
I20260929 10:54:29.022904 11245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 512de9c5d1ac47febcbdfed527afb630: Bootstrap starting.
I20260929 10:54:29.023727 11245 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 512de9c5d1ac47febcbdfed527afb630: Neither blocks nor log segments found. Creating new log.
I20260929 10:54:29.024039 11245 log.cc:824] T 00000000000000000000000000000000 P 512de9c5d1ac47febcbdfed527afb630: Log is configured to *not* fsync() on all Append() calls
I20260929 10:54:29.024459 11363 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:54:29.027956 11326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:29.004459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41400 (local address 127.20.80.252:37429)
0929 10:54:29.004557 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:29.004559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:29.013691 (+ 9132us) server_negotiation.cc:407] Connection header received
0929 10:54:29.013799 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:29.013800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:29.013810 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:29.013840 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:29.014655 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:29.015344 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:29.016382 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:29.016681 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:29.016723 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:29.018822 (+ 2099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:29.018824 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:29.018824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:29.018831 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:29.026270 (+ 7439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:29.026822 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:29.026824 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:29.026824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:29.026894 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:29.027191 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:29.027211 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:29.027211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:29.027340 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:29.027437 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:29.027765 (+ 328us) server_negotiation.cc:299] Negotiation successful
0929 10:54:29.027861 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 10:54:29.035249 11245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 512de9c5d1ac47febcbdfed527afb630: No bootstrap required, opened a new log
I20260929 10:54:29.037773 11245 raft_consensus.cc:348] T 00000000000000000000000000000000 P 512de9c5d1ac47febcbdfed527afb630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e94d1118144dd0b0b685473197984e" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 40567 } } peers { permanent_uuid: "512de9c5d1ac47febcbdfed527afb630" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 33893 } } peers { permanent_uuid: "dcaf5a98737c4086927dcc5fcc7f5d31" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37429 } }
I20260929 10:54:29.037940 11245 raft_consensus.cc:729] T 00000000000000000000000000000000 P 512de9c5d1ac47febcbdfed527afb630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 512de9c5d1ac47febcbdfed527afb630, State: Initialized, Role: FOLLOWER
I20260929 10:54:29.038043 11245 consensus_queue.cc:260] T 00000000000000000000000000000000 P 512de9c5d1ac47febcbdfed527afb630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e94d1118144dd0b0b685473197984e" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 40567 } } peers { permanent_uuid: "512de9c5d1ac47febcbdfed527afb630" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 33893 } } peers { permanent_uuid: "dcaf5a98737c4086927dcc5fcc7f5d31" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37429 } }
I20260929 10:54:29.038345 11368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 512de9c5d1ac47febcbdfed527afb630 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e94d1118144dd0b0b685473197984e" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 40567 } } peers { permanent_uuid: "512de9c5d1ac47febcbdfed527afb630" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 33893 } } peers { permanent_uuid: "dcaf5a98737c4086927dcc5fcc7f5d31" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37429 } } }
I20260929 10:54:29.038431 11245 sys_catalog.cc:564] T 00000000000000000000000000000000 P 512de9c5d1ac47febcbdfed527afb630 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:54:29.038450 11368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 512de9c5d1ac47febcbdfed527afb630 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:54:29.038648 11249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:29.017215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53962 (local address 127.20.80.253:33893)
0929 10:54:29.017258 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:29.017260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:29.028674 (+ 11414us) server_negotiation.cc:407] Connection header received
0929 10:54:29.028802 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:29.028803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:29.028824 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:29.028854 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:29.029957 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:29.030712 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:29.032145 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:29.032528 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:29.032572 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:29.034970 (+ 2398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:29.034971 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:29.034972 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:29.034990 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:29.036645 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:29.037272 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:29.037273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:29.037274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:29.037347 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:29.037635 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:29.037636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:29.037636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:29.037776 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:29.037875 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:29.038467 (+ 592us) server_negotiation.cc:299] Negotiation successful
0929 10:54:29.038548 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 10:54:29.038846 11219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f6e94d1118144dd0b0b685473197984e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "512de9c5d1ac47febcbdfed527afb630"
I20260929 10:54:29.038954 11219 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 512de9c5d1ac47febcbdfed527afb630 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:54:29.039944 11219 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 512de9c5d1ac47febcbdfed527afb630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e94d1118144dd0b0b685473197984e in term 1.
W20260929 10:54:29.040705 11380 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 512de9c5d1ac47febcbdfed527afb630: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:54:29.040766 11380 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:54:29.043495 11172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:29.018509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47074 (local address 127.20.80.254:40567)
0929 10:54:29.018632 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:29.018634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:29.022953 (+ 4319us) server_negotiation.cc:407] Connection header received
0929 10:54:29.023062 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:29.023063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:29.023076 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:29.023104 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:29.023832 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:29.024532 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:29.033086 (+ 8554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:29.033405 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:29.033453 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:29.040256 (+ 6803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:54:29.040258 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:54:29.040258 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:54:29.040267 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:54:29.042034 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:29.042423 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:29.042424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:29.042424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:29.042478 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:54:29.042703 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:54:29.042704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:54:29.042704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:54:29.042847 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:54:29.042936 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:29.043316 (+ 380us) server_negotiation.cc:299] Negotiation successful
0929 10:54:29.043401 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260929 10:54:29.044557 11295 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dcaf5a98737c4086927dcc5fcc7f5d31 [term 1 FOLLOWER]: Refusing update from remote peer f6e94d1118144dd0b0b685473197984e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:54:29.044575 11219 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 512de9c5d1ac47febcbdfed527afb630 [term 1 FOLLOWER]: Refusing update from remote peer f6e94d1118144dd0b0b685473197984e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:54:29.044872 11361 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f6e94d1118144dd0b0b685473197984e [LEADER]: Connected to new peer: Peer: permanent_uuid: "dcaf5a98737c4086927dcc5fcc7f5d31" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:54:29.044998 11329 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f6e94d1118144dd0b0b685473197984e [LEADER]: Connected to new peer: Peer: permanent_uuid: "512de9c5d1ac47febcbdfed527afb630" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 33893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:54:29.046160 11368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 512de9c5d1ac47febcbdfed527afb630 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6e94d1118144dd0b0b685473197984e. Latest consensus state: current_term: 1 leader_uuid: "f6e94d1118144dd0b0b685473197984e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e94d1118144dd0b0b685473197984e" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 40567 } } peers { permanent_uuid: "512de9c5d1ac47febcbdfed527afb630" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 33893 } } peers { permanent_uuid: "dcaf5a98737c4086927dcc5fcc7f5d31" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37429 } } }
I20260929 10:54:29.046265 11368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 512de9c5d1ac47febcbdfed527afb630 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:54:29.046808 11343 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcaf5a98737c4086927dcc5fcc7f5d31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6e94d1118144dd0b0b685473197984e. Latest consensus state: current_term: 1 leader_uuid: "f6e94d1118144dd0b0b685473197984e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e94d1118144dd0b0b685473197984e" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 40567 } } peers { permanent_uuid: "512de9c5d1ac47febcbdfed527afb630" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 33893 } } peers { permanent_uuid: "dcaf5a98737c4086927dcc5fcc7f5d31" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37429 } } }
I20260929 10:54:29.046902 11343 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcaf5a98737c4086927dcc5fcc7f5d31 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:54:29.049948 11329 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6e94d1118144dd0b0b685473197984e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f6e94d1118144dd0b0b685473197984e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e94d1118144dd0b0b685473197984e" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 40567 } } peers { permanent_uuid: "512de9c5d1ac47febcbdfed527afb630" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 33893 } } peers { permanent_uuid: "dcaf5a98737c4086927dcc5fcc7f5d31" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37429 } } }
I20260929 10:54:29.050045 11329 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6e94d1118144dd0b0b685473197984e [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:29.050123 11343 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcaf5a98737c4086927dcc5fcc7f5d31 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f6e94d1118144dd0b0b685473197984e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e94d1118144dd0b0b685473197984e" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 40567 } } peers { permanent_uuid: "512de9c5d1ac47febcbdfed527afb630" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 33893 } } peers { permanent_uuid: "dcaf5a98737c4086927dcc5fcc7f5d31" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37429 } } }
I20260929 10:54:29.050182 11367 mvcc.cc:205] Tried to move back new op lower bound from 7334622286004174848 to 7334622285870178304. Current Snapshot: MvccSnapshot[applied={T|T < 7334622286004174848}]
I20260929 10:54:29.050218 11343 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcaf5a98737c4086927dcc5fcc7f5d31 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:54:29.050467 11384 mvcc.cc:205] Tried to move back new op lower bound from 7334622286004174848 to 7334622285870178304. Current Snapshot: MvccSnapshot[applied={T|T < 7334622286004174848}]
I20260929 10:54:29.050475 11363 catalog_manager.cc:1347] Generated new cluster ID: 859c8b3b820946da9a252a5af01ecfed
I20260929 10:54:29.050518 11363 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:54:29.058535 11361 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6e94d1118144dd0b0b685473197984e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f6e94d1118144dd0b0b685473197984e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e94d1118144dd0b0b685473197984e" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 40567 } } peers { permanent_uuid: "512de9c5d1ac47febcbdfed527afb630" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 33893 } } peers { permanent_uuid: "dcaf5a98737c4086927dcc5fcc7f5d31" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37429 } } }
I20260929 10:54:29.058645 11361 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6e94d1118144dd0b0b685473197984e [sys.catalog]: This master's current role is: LEADER
I20260929 10:54:29.059099 11382 mvcc.cc:205] Tried to move back new op lower bound from 7334622286004174848 to 7334622285870178304. Current Snapshot: MvccSnapshot[applied={T|T < 7334622286004174848}]
I20260929 10:54:29.059188 11368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 512de9c5d1ac47febcbdfed527afb630 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f6e94d1118144dd0b0b685473197984e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e94d1118144dd0b0b685473197984e" member_type: VOTER last_known_addr { host: "127.20.80.254" port: 40567 } } peers { permanent_uuid: "512de9c5d1ac47febcbdfed527afb630" member_type: VOTER last_known_addr { host: "127.20.80.253" port: 33893 } } peers { permanent_uuid: "dcaf5a98737c4086927dcc5fcc7f5d31" member_type: VOTER last_known_addr { host: "127.20.80.252" port: 37429 } } }
I20260929 10:54:29.059259 11368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 512de9c5d1ac47febcbdfed527afb630 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:54:29.074246 11363 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:54:29.075114 11363 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:54:29.081118 11363 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6e94d1118144dd0b0b685473197984e: Generated new TSK 0
I20260929 10:54:29.081377 11363 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 10:54:29.109519 11172 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:54:29.107748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47090 (local address 127.20.80.254:40567)
0929 10:54:29.107792 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:29.107795 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:29.108800 (+ 1005us) server_negotiation.cc:407] Connection header received
0929 10:54:29.108914 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:29.108915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:29.108927 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:29.108960 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:29.109447 (+ 487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47090: BlockingRecv error: recv got EOF from 127.0.0.1:47090 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260929 10:54:29.110499 11249 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:54:29.108164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53968 (local address 127.20.80.253:33893)
0929 10:54:29.108217 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:29.108219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:29.109946 (+ 1727us) server_negotiation.cc:407] Connection header received
0929 10:54:29.110048 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:29.110049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:29.110060 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:29.110089 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:29.110434 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53968: BlockingRecv error: recv got EOF from 127.0.0.1:53968 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260929 10:54:29.111513 11326 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:54:29.108526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41412 (local address 127.20.80.252:37429)
0929 10:54:29.108566 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:29.108567 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:29.110869 (+ 2302us) server_negotiation.cc:407] Connection header received
0929 10:54:29.111045 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:29.111046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:29.111057 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:29.111088 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:29.111451 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41412: BlockingRecv error: recv got EOF from 127.0.0.1:41412 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260929 10:54:29.113932 11249 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:54:29.113240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53976 (local address 127.20.80.253:33893)
0929 10:54:29.113283 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:29.113285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:29.113425 (+ 140us) server_negotiation.cc:407] Connection header received
0929 10:54:29.113531 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:29.113532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:29.113552 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:29.113581 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:29.113885 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53976: BlockingRecv error: recv got EOF from 127.0.0.1:53976 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260929 10:54:29.117112 11249 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:54:29.116292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53984 (local address 127.20.80.253:33893)
0929 10:54:29.116327 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:29.116329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:29.116558 (+ 229us) server_negotiation.cc:407] Connection header received
0929 10:54:29.116656 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:29.116657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:29.116668 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:29.116698 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:29.117064 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53984: BlockingRecv error: recv got EOF from 127.0.0.1:53984 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260929 10:54:29.121411 11249 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:54:29.120578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53988 (local address 127.20.80.253:33893)
0929 10:54:29.120615 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:29.120616 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:29.120858 (+ 242us) server_negotiation.cc:407] Connection header received
0929 10:54:29.120958 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:29.120959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:29.120970 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:29.121021 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:29.121365 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53988: BlockingRecv error: recv got EOF from 127.0.0.1:53988 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260929 10:54:29.127610 11249 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:54:29.126738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53992 (local address 127.20.80.253:33893)
0929 10:54:29.126776 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:29.126778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:29.127019 (+ 241us) server_negotiation.cc:407] Connection header received
0929 10:54:29.127145 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:29.127146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:29.127158 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:29.127189 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:29.127558 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53992: BlockingRecv error: recv got EOF from 127.0.0.1:53992 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260929 10:54:29.139015 11249 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:54:29.138049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54002 (local address 127.20.80.253:33893)
0929 10:54:29.138094 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:29.138096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:29.138398 (+ 302us) server_negotiation.cc:407] Connection header received
0929 10:54:29.138519 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:29.138520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:29.138534 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:29.138566 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:29.138954 (+ 388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54002: BlockingRecv error: recv got EOF from 127.0.0.1:54002 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260929 10:54:29.157727 11249 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:54:29.156584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54008 (local address 127.20.80.253:33893)
0929 10:54:29.156631 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:29.156633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:29.156918 (+ 285us) server_negotiation.cc:407] Connection header received
0929 10:54:29.157094 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:29.157095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:29.157108 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:29.157139 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:29.157663 (+ 524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54008: BlockingRecv error: recv got EOF from 127.0.0.1:54008 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260929 10:54:29.192112 11249 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:54:29.191302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54012 (local address 127.20.80.253:33893)
0929 10:54:29.191358 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:29.191361 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:29.191474 (+ 113us) server_negotiation.cc:407] Connection header received
0929 10:54:29.191550 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:29.191552 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:29.191564 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:29.191611 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:29.192037 (+ 426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54012: BlockingRecv error: recv got EOF from 127.0.0.1:54012 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260929 10:54:29.258096 11249 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:54:29.257251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54022 (local address 127.20.80.253:33893)
0929 10:54:29.257334 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:29.257337 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:29.257442 (+ 105us) server_negotiation.cc:407] Connection header received
0929 10:54:29.257563 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:29.257564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:29.257576 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:29.257626 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:29.258049 (+ 423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54022: BlockingRecv error: recv got EOF from 127.0.0.1:54022 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260929 10:54:29.388077 11249 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:54:29.387360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54032 (local address 127.20.80.253:33893)
0929 10:54:29.387432 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:29.387434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:29.387560 (+ 126us) server_negotiation.cc:407] Connection header received
0929 10:54:29.387601 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:29.387602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:29.387615 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:29.387716 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:29.387989 (+ 273us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54032: BlockingRecv error: recv got EOF from 127.0.0.1:54032 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260929 10:54:29.645953 11249 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:54:29.645294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54036 (local address 127.20.80.253:33893)
0929 10:54:29.645366 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:29.645368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:29.645378 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:54:29.645432 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:29.645433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:29.645443 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:29.645489 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:54:29.645889 (+ 400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54036: BlockingRecv error: recv got EOF from 127.0.0.1:54036 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 10:54:29.999692 11354 catalog_manager.cc:1259] Loaded cluster ID: 859c8b3b820946da9a252a5af01ecfed
I20260929 10:54:29.999807 11354 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dcaf5a98737c4086927dcc5fcc7f5d31: loading cluster ID for follower catalog manager: success
I20260929 10:54:30.000691 11354 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dcaf5a98737c4086927dcc5fcc7f5d31: acquiring CA information for follower catalog manager: success
I20260929 10:54:30.001091 11354 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dcaf5a98737c4086927dcc5fcc7f5d31: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 10:54:30.041254 11380 catalog_manager.cc:1259] Loaded cluster ID: 859c8b3b820946da9a252a5af01ecfed
I20260929 10:54:30.041365 11380 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 512de9c5d1ac47febcbdfed527afb630: loading cluster ID for follower catalog manager: success
I20260929 10:54:30.042196 11380 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 512de9c5d1ac47febcbdfed527afb630: acquiring CA information for follower catalog manager: success
I20260929 10:54:30.042505 11380 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 512de9c5d1ac47febcbdfed527afb630: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 10:54:30.162084 11453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:30.159115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54050 (local address 127.20.80.253:33893)
0929 10:54:30.159249 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:30.159251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:30.159263 (+ 12us) server_negotiation.cc:407] Connection header received
0929 10:54:30.159396 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:30.159397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:30.159449 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:30.159531 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:54:30.159762 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:30.160283 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:30.161199 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:30.161406 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:30.161468 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:30.161652 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:54:30.161801 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:54:30.161878 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:30.161962 (+ 84us) server_negotiation.cc:299] Negotiation successful
0929 10:54:30.161986 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":65,"threads_started":1}
I20260929 10:54:30.166307 11460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:30.164055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41414 (local address 127.20.80.252:37429)
0929 10:54:30.164163 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:30.164164 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:30.164172 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:54:30.164208 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:30.164208 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:30.164242 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:30.164312 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:54:30.164395 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:30.164856 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:30.165598 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:30.165801 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:30.165854 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:30.165986 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:54:30.166086 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:54:30.166142 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:30.166179 (+ 37us) server_negotiation.cc:299] Negotiation successful
0929 10:54:30.166196 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":53,"threads_started":1}
I20260929 10:54:30.170789 11467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:30.167974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47092 (local address 127.20.80.254:40567)
0929 10:54:30.168163 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:30.168165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:30.168174 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:54:30.168217 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:30.168218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:30.168282 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:30.168364 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:54:30.168476 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:30.169054 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:30.169826 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:30.170055 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:30.170170 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:30.170364 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:54:30.170476 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:54:30.170532 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:30.170592 (+ 60us) server_negotiation.cc:299] Negotiation successful
0929 10:54:30.170617 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":59,"threads_started":1}
I20260929 10:54:30.170900 11453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:30.167969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54056 (local address 127.20.80.253:33893)
0929 10:54:30.168010 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:30.168013 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:30.168228 (+ 215us) server_negotiation.cc:407] Connection header received
0929 10:54:30.168296 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:30.168297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:30.168308 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:30.168380 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:54:30.168528 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:30.169063 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:30.170120 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:30.170410 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:30.170476 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:30.170628 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:54:30.170662 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:54:30.170691 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:30.170783 (+ 92us) server_negotiation.cc:299] Negotiation successful
0929 10:54:30.170806 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 10:54:30.171530 11460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:54:30.168147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41424 (local address 127.20.80.252:37429)
0929 10:54:30.168182 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0929 10:54:30.168184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:54:30.168196 (+ 12us) server_negotiation.cc:407] Connection header received
0929 10:54:30.168284 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:54:30.168285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:54:30.168294 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:54:30.168325 (+ 31us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:54:30.169047 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:30.169763 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:30.170747 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:54:30.171013 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:54:30.171074 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:54:30.171251 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:54:30.171282 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:54:30.171312 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0929 10:54:30.171434 (+ 122us) server_negotiation.cc:299] Negotiation successful
0929 10:54:30.171457 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 10:54:30.172590 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 11102
I20260929 10:54:30.178225 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 11177
I20260929 10:54:30.183162 20803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfQc4Oq/build/release/bin/kudu with pid 11252
2026-09-29T10:54:30Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2842 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5656 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (274063 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST